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ABSTRACT 


This study develops a computer assisted instruction 
response evaluation technique to aid CAI lesson writers in 
course development. The objectives for this technique was 
to experimentally identify principal incorrect response 
forms and determine characteristic error patterns for a 
given concept to be taught. Using a specific concept, the 
proposed technique was evaluated. Sixty-one subjects were 
randomly selected from three different universities and 
asSigned to solve a problem-solving task. Using mental 
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protocol for this experiment. Although the response eval- 
uation technique developed in this study was reasonably 
Successful, a specific response error pattern for each 
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I. INTRODUCTION 


Abrams (1971) asserts that the most difficult problems 
impeding the progress of computer-assisted instruction (CAI) 
are those involving the basics of the educational process 
itself and the inherent complexity of CAI from the point of 
view of the educators. While a good CAI system will not 
appear complex to the students and the instructors who use 
it, the existing and proposed new system are difficult for 
even knowledgeable educators to evaluate for applicability 
and desirability. 

Even more basic is the problem of the use of diverse 
forms of responses to indicate the attainment of knowledge 
and understanding. This problem is associated with two 
questions: 

(1) What type of response is required from the 
student and the CAI system for a given 
educational evolution? 

(2) Will this response be a true indicator of 
the student's understanding of the concepts 
involved? 

These are difficult, nevertheless, important questions as 
indicated by Simon: "the possibilities that are opened up 
for improving human problem-solving, thinking, and decision- 
making activities" are "at least as important" as the 
genuine revolutionary "developments [which are now apparent] 


in automation." [Weissinger-Baylon, 173, p. 1] : 





The author shares both Abram'’s concern and Simon's con- 
fidence for this intrinsic problem area of computer-assisted 
instruction. Although much has been accomplished in this 
problematical area, much remains to be solved. Until the 
many factors involved in the response evaluation process is 
more fully explored and more thoroughly understood, it is 
the author's opinion that it will be difficult for CAI to 
realize its full potential. 

There have been many variations in the definition of 
computer-assisted instruction (CAI) in the past two decades. 
The one definition that encompasses the whole field and 
which will be used in this paper is best defined by Abrams: 

. - . the use of computers to aid teachers and students in 

the educational process, utilizing such functions as pre- 

senting materials and problems, guiding a student's path 

through a course by selecting the material to be pre- 

sented or by assigning tasks to be completed away from the 

computer, or any combination of these functions. 

(Abrams, do Lee De 4} 

Therefore, CAI does not just simply imply typing out 
questions to which students respond and by then giving cues 
and different hits, CAI includes the computational use of 
the computer, when appropriate, as part of the instruction. 

Computer-assisted instruction systems may be generally 
viewed in two ways: machine-directed and student-directed. 
In the machine-directed approach, various paths and alterna- 


tives through instructional material is programmed into a 


machine. In this approach there must be prior knowledge of 
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the needs of the student and a predetermined method by which 
the computer can evaluate the student's current state of 
knowledge. In addition, the student's past performance 
record and techniques for describing a further path for each 
student, which would optimize his learning experience, must 
be at the machine's disposal. 

In the student-directed approach, the sequence of the 
material presented is altered only at the request of the 
student. The approach uses the aspect of learning by dis- 
covery and experimentation. This area, however, has been 
almost untouched by current research and, therefore, much 
less has been done to utilize this type of approach. 

This paper deals primarily with the machine-directed CAI 
approach. A new computer-assisted instruction (CAI) tech- 
nique with the potential to improve the effectiveness of 
CAI was developed and evaluated. This technique experi- 
mentally identifies the principal incorrect forms of any 
given concept that is taught. Characteristic error patterns 
are associated with incorrect forms so that the subject may 
be presented with a CAI response tailored to that particular 
incorrect concept that he has acquired. New instructional 
material or other techniques could then be utilized depend- 
ing upon the student's understanding or lack of understand- 
ing of specific concepts. 

Absence of any contributing literature in the area of 


CAI response evaluation and personal communications with 


seal 





prominent individuals in the area of CAI, indicate that the 
approach discussed in this study, is the first significant 
attempt to provide a specific set of rules or guidelines 

for evaluating the subject's responses. This does not im- 

ply that the response evaluation area of CAI was never con- 
Sidered or discussed, but that a set of rules has never 

been formalized. The following method has therefore been 
designed to provide CAI lesson writers with an additional tool 


for structuring CAI course material. 
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IIT. BACKGROUND 


A. THE NEED FOR A RESPONSE EVALUATION TECHNIQUE 

The use of various computer-assisted instruction systems 
has been on the upswing in the last decade. This growth has 
been desirable and is a result of the increased amount of 
human knowledge that needs to be attained, the ever increas- 
ing number of students to be taught, and the availability of 
qualified instructors. Early studies revealed that consist- 
ent time savings occurred during a three year experiment 
using microcomputers to manage students‘ self-paced pro- 
gression through a course (Evans and Johns, 1979). Orlansky 
and String (1979), in an iconoclastic attack on the cost- 
effectiveness of thirty studies of military computer-based 
education (CBE) have also concluded that a thirty percent 
time savings is possible. It can therefore be generally 
agreed that CAI can achieve educational objectives more effi- 
Ciently and more effectively than other teaching methods. 
However, four basic educational factors accentuate the need 
for a computer-assisted instruction (CAI) response evalua- 
tion technique. 

The first factor is the trend toward individualized 
instruction. The fewer the number of students for which a 
teacher is responsible, the more time and individual 


attention each student will receive. Therefore, from a 
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theoretical aspect, the more each student will gain from his 
educational experience. Conventional mass media such as 
books, films, television, and even programmed instruction 
are inherently incapable of individualization. Books, films, 
and television programs would have to be rewritten to fit 
each student, and conventional programmed instruction systems 
lack the flexibility and decision-making capability required 
of a truly effective individualized system. CAI seems to 
have the potential to meet these requirements. However, the 
current relatively sophisticated computer-assisted instruc- 
tion systems are only tailored towards students who are 
homogeneous in background, ability, cognitive style, and 
motivation. The systems are incapable of individualization 
in the sense of structuring a reSponse as a true indicator 
of the student's understanding of the concepts involved. 
However, by evaluating the subject response and then pro- 
viding a tailor-made CAI response to reflect understanding 
or lack of understanding of a given concept, seems to have 
the potential to improve the effectiveness of CAI and its 
value to individualized instruction. 

The second factor is the evaluation of the student's 
learning progress. Although, sophisticated CAI systems 
keep track of the number of wrong responses for a particu- 
lar concept, they do not identify to the educator the 
incorrect concept that the student learns. Using a response 


evaluation technique, the wrong concept may be identified 
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and new instructional material, review material, drill and 
practice sessions, tutoring, or other techniques could then 
be utilized depending on the student's understanding. 

The trend towards the use of microcomputers is the third 
factor. Computer-assisted instruction has typically been 
applied using large, dedicated computer systems and special 
CAI languages. There are virtually no limitation on memory 
or file sizes for the instructional material. However, 
microcomputers are now being sold at prices comparable to 
color television sets and their use in educational settings 
will skyrocket in the next few years. Invariably, CAI will 
be one of the uses to which these computer systems are put. 
These systems, however, have inherent limitations. The most 
important being memory size. Since, these limitations are 
fixed attributes of the systems, they must be dealt with 
creatively. One method is to use a response evaluation 
technique. Using such a technique, the size of the programs 
may be reduced by limiting the required number of paths or 
alternatives through the instructional material. 

Finally, the fourth factor is the limited applicability 
of computer-assisted instruction to university level subjects. 
Weissinger-Baylon asserts that according to Atkinson, "the 
relatively sophisticated computer-assisted instruction 
Systems that have been successfully introduced for reading 
and elementary areas are neither applicable to under- 
graduate mathematics nor to other university level subjects; 


too little is known of the structure of the required skills 
) 





or of the psychological processes for their mastery." Until 
the type of response for certain concepts are more fully 
explored and more thoroughly understood it will be difficult 


for CAI to realize its full potential. 


B. RESEARCH OBJECTIVES 

The most difficult problem impeding the progress of com- 
puter-assisted instruction are those involving the basics of 
the educational process itself. The type of response that 
must be obtained from the student to get an indicator of his 
true understanding of the concept, is a difficult question. 
Although there has been some work performed by Newell, Simon, 
Shaw, and Weissinger~Baylon in the area of human problem 
solving |Weissinger-Baylon, 1978, Pp. |, it has only been 
applied to the area of computer-assisted instruction in a 
very limited amount. Substantial progress in this direction, 
however, will permit far reaching new approaches to education. 

The present study is conceived as one step in a longer 
program, the development of major improvements, efficiency 
and value of computer-assisted instruction systems. The 
immediate objective is to develop and evaluate a new com- 
puter~-assisted instruction (CAI) technique uSing mental 
imagery and verbal responses to experimentally identify the 
principal incorrect forms of a given concept that the stu- 
dents usually learns. Characteristic error patterns will be 
identified and associated with incorrect concept forms on a 
diagnostic test and will be evaluated in regards to its 


application to a computer-~assisted instruction system. 
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C. TASK SELECTION 

Task selection for the development and evaluation of any 
new technique or procedure must be based on many factors and 
therefore makes it a difficult chore. Weissinger-Baylon, 
from his research, States that the most important factors in 
task selection are: “emphasis on tractability or support 
for the experimenters hypothesis must be minimized; the re- 
search must have application, at least potentially; and 
finally, the complexity of the task must not limit the 
communicability of the findings." The author agrees with 
Weissinger-Baylon's findings, but adds that a task may be 
selected because of it's interest to the analyst and because 
1t complements the work of others. 

The choice for the selection of mathematics as a field 
to test and evaluate responses for a CAI system was made 
as a result of its many advantages: "for the chosen problem 
area, both concept and vocabulary are highly developed; ade- 
gGuate intellectual resources are available for this topic; 
its method and administrative procedures are convenient; 
and it iS an important area of application, controlling al- 
most all entries to attractive professions." 

Determining a problem area within the chosen field was 
the next stage in the selection process. It was a difficult 
task. The material had to satisfy both the requirements of 
meeting the objectives of this research and, most impor- 


tantly, be within the grasp of existing information 
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processing methodologies. 

Weissinger-Baylon's personal communication with Karl 
deLeeuw [Weissinger-Baylon, oo, Pp. 30] , proposed an 
analysis of a group of closely related concepts having to 
do with "bounding" of sets. It was felt such a concept 
would adequately propose or illustrate, at least theoreti- 
cally, the technique to evaluate and analyze responses for 
a CAI system. The problem that arose, however, was what 
type of bounding concept to use--the greatest lower bound, 
the least upper bound, or determine if the set is bounded 
above or below. 

Preliminary studies conducted by Weissinger-Baylon on 
the proposed concepts, suggested that the focus of the 
analysis for a response evaluation process for a CAI 
System, be narrowed to a single definition. His-studies 
on the upper bound concept produced the most interesting 
results and it was determined that the importance of the 
upper bound concept, its manageability, and Weissinger- 
Baylon's prior research in this area, made this concept 
the most attractive candidate for this research study in 


evaluating the response process for a CAI system. 


D. METHODOLOGICAL OVERVIEW 
The development of a suitable methodology proceeded in 


stages. Test results from a research study conducted by 
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Weissinger-Baylon on the upper-bound of sets concept, were 
made available for this research, and pre-empted the need 


for duplicate and redundant testing in the initial phase of 


the study. 
1. Subjects 


The sixty-one students who served as subjects were 
volunteers from three different universities: the Naval 
Postgraduate School, Stanford University and Colorado State 
University. Subjects were selected to provide a reasonable 
balanced representation by sex, specialization and profes- 
Sional levels. They were obtained from bulletin board 
announcements, distribution of sign-up sheets and by 
personal request. With the exception of the Stanford Uni- 
versity students, who received a $6.00 fee, no other 
compensation was offered for the subject's participation in 
this experiment. The mix consisted of 23% females and 77% 
males. Of these, 33% were graduate or undergraduate stu- 
dents in the mathematical sciences (Computer science, 
Statistics, Operations research and Mathematical sciences) 
and 67% were undergraduates or non-mathematical science 
graduates. Prior research had dictated that subjects with- 
out university mathematics would experience difficulty with 
this particular concept and were therefore required to have 


had at least two years of calculus. 
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Ze Design 


The problem-solving task was initiated by looking 
at fifteen different, but related definitions from advanced 
calculus. From the results, a final version was produced, 
which focused on one definition: the upper bound of a set 
of real numbers. The criteria for an upper bound of a set 
was met if there existed a real number which was greater 
than every element of the set. 

Nine problems were chosen from standard texts of ad- 
vanced calculus and grouped into the following four types 
of sets: 

(1) finite sequences: e.g. {1,2,3,4! 

(2) intervals: e.g. (a,b) 


(3) convergent infinite sequences: 
Sicp lle / 2/3, ...i/n,..<t 


(4) divergent infinite sequences: 
SGn roo, .. Nips. st 


[Weissinger-Baylon, Eos. 32] 
In addition, the above sets were presented in the following 
methods: | 
(1) enumerative: {1,2,3,4} 
(2) nominal: N, the natural numbers 
(3) attributive: finite set 
[Weissinger-Baylon, 1978, p. 32] 
The sets were randomized by type and presentation method. 
These representations vary slightly from the final de- 


Sign since they were modified to make them suitable for a 
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wide range of subjects. Further, it is well to note that 
the degree of accuracy will be based on the degree to which 
the subject's responses match the existing textbooks. The 
production of imagery or step by step analogy of the prob- 


lem set was not consciously a criterion. 


3. Procedure 

The detailed, step-by-step procedure for the CAI 
response evaluation technique is given in chapter 3. The 
procedural model, as designed, consists of seven major 
components: selection of target content, design of 
problem-solving task, selection of protocol methodology, 
pretesting, refining the task, administering the final 
version of the task, and analysis of results. 

The procedural methodology that was used for adminis- 
tering the problem-solving task consisted of contacting 
the subject prior to the experiment and agreeing on a set 
time for administration of the task. Before beginning the 
problem-solving task, the subjects were interviewed to 
determine their qualifications for testing on the 
particular concept. In addition, during the interview the 


subjects provided the following information: year, 
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department, and educational and career plans. When the 
participant was found to be qualified, he was given the 
following instructions: 
Instructions. On the next page, you will find a defini- 
tion taken from an undergraduate mathematics course. At 
first the language may seem impoSing, but the concept is 
not difficult. Please read carefully, and try to under- 
Stand it as well as you can. Next, go on to the practice 
problem and the questions which follow: 
We would like you to think out loud as much as possible. 
If you use any mental imagery in your thinking, please 
draw or describe it. At the end of each problem, you 
are to write a complete summary of your thought process. 
You may begin when you wish. There is no time limit. 
Please ask any questions which you feel are necessary. 
Do not forget to think out loud, or draw or describe any 
mental imagery you may have. 
\Weissinger-Baylon, oo 34 
A tape recorder was then put in front of the subject to 
report his oral responses, if any occurred. When the 
subject indicated that he understood the instructions, he 
was given the test booklet and then asked to start the 
exam. The examiner was seated beside the subject to 
observe and take notes. At the end of the entire Session, 
the subject waS again interviewed to clarify and explain 
the exam and, in some cases, uncover errors in the written 
protocol without the risk of biasing the responses. 
In summary, the administration process was designed to 
Systematically obtain data using the following protocol 


methodology: 


1. Verbal protocol--to clarify infrequent ambiguities 
in the written summaries. 


ee 


2. Graphic protocol--to illustrate a path of the 
subject's thought processes. 


3. Written summaries--to obtain a formalized, step-by 
step analogy of the problem. 


4. Interaction with experimenter--to obtain additional 
clarification when imagery explanations were 
ambiguous and when steps were omitted during the 
written summaries. 

The importance of this data collection method is demon- 


strated in the data analysis phase of the procedural model. 
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III. DEVELOPMENT OF A RESPONSE EVALUATION TECHNIQUE 


A response iS a unit of behavior and the building block 
of complex performances. The flick of an eye and the twitch 
of a finger are examples of simple responses; eating, walk- 
ing, speaking and reading are all instances of more complex 
responses. A primary objective of educational technology 
is the guidance of individual's responses. To accomplish 
this objective, the educator must first define and enumer- 
ate the one: of the performance, that is, the 
responses, that he wishes to produce. It is then possible 
to arrange the stimulus conditions which will result in the 
desired response. It also becomes possible to develop 
objective measures of the frequency and accuracy of the 
response. 

The consequences of a student's response are extremely 
important in learning. The events which follow the occur- 
rence of a response have an effect upon future behavior. 
Examples of such response consequences are reward, punish- 
ment, and knowledge of results. The reward or other stimu- 
lation provided to the learner immediately after responding 
is a Significant factor which seems to determine whether 
learning takes place. The occurrence of certain consequen- 
ces of behavior that are effective in producing and 
maintaining behavior is called reinforcement by behavioral 


psychologists. In an educational situation, the instructor 
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controls the consequences of the student's behavior. Since 
these consequences determine whether the student learns, 
the educator will want to maximize those consequences which 
facilitate learning. 

Definition of an effective and efficient procedural 
model for a response evaluation methodology began with an 
analysis of the characteristics of the instructional environ- 
ment. This examined specifically the environment defined 
by the evaluation of course mastery, and the types of in- 
Sstructional problems for which a response evaluation process 
could provide solutions in this environment. 

The procedural model, as derived, consists of seven 
major components, each with a number of constituent steps. 
The components are selection of target content, design of 
problem solving task, selecting protocol methodology, pre- 
testing, refining the task, administering the final version 
of the task, and analysis of results. The full model is 
shown in figure 1. The model's seven component steps are 


described in the following paragraphs. 


A. SELECTION OF TARGET CONTENT 

The first component of the procedural model (see figure 
2) assumes the presence of a main track of non-CAI materials 
and defines criteria for selecting those materials for which 
alternative CAI modules are to be developed. The steps in 


this component do not constitute a precise algorithm for 
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SELECTION OF TARGET CONTENT 


1. Calculus 
2. Upper bound of Sets 






DESIGN TEST 
1. Preliminary Knowledge of System and Problem Solving Task 
2. Determine All possible responses that are to be achieved 


a. carrect 
b. incorrect 














SELECT PROTOCOL METHODOLOGY 


Subjects to draw visual Mental Imagery 
Subjects to use verbal responses 





PRETEST: 12 Subjects 


MODIFY TEST VERSION 


1. Delete Problems with more than one theoretical response 
2. Delete Problems with a multitude of incorrect responses 








ADMINISTER FINAL VERSION OF THE TEST 


1. Sixty-one subjects 
2. Balance subjects by abilit 


ANALYZE RESULTS 


FIGURE 1. This flowchart summarizes the main steps in the 
experimental design and administration of a 
response evaluation procedure. 
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content selection. Rather the intent is to identify a 
number of factors which should be considered in determining 
target content selection. Few hard and fast rules can be 
defined, and value judgements play a major role in the 
decision process. 

The first step in the selection of target content is to 
select a course or courses where the use of CAI will be 
beneficial to the training process. This should be a 
school-level decision based on considerations as training 
cost per student, instructor student ratios, and types of 
training materials used. 

The principal consideration must be to choose a course 
where CAI materials can improve motivation, and where the 
unique capabilities of CAI will provide needed information. 

After selecting a course, the next step is to select a 
specific concept. Here, the selection considerations are 
the importance of the particular concept, concept failure 
rates, variability of concept mastery test scores, and aver- 
age times and variability of times to complete the concept. 

When the two selection steps (course and concept) are 
completed, it must be determined if CAI is feasible for the 
tentative selections. This is where preliminary knowledge 
of the system plays an important role. Possible reasons for 
determining non-feasibility of CAI include necessity for 
graphics or simulations which are beyond the capabilities of 


the available system and personnel, or too few concept 
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selections to justify hardware or development cost. 

With the final determination of CAI feasibility, the 
educator is now able to define and enumerate the components 
of performance, in other words, the response he wishes to 
produce for a given task. The educator is ready to begin 
the next component of the procedural model, the design of 


the problem-solving task. 


B. DESIGN OF PROBLEM SOLVING TASK 

The steps of the design and the development of a 
problem-solving task component is shown in figure 3. It 
assumes that the course and concept selection together with 
their CAI feasibility have already been determined. 

In starting to develop a new teaching session, an edu- 
cator must be given a certain amount of basic preliminary 
information about the material and system to be used. This 
may be provided in the form of text, drawings, diagrams, 
moving pictures, or any combination of these modes. This 
process will ensure that the finished materials avoid such 
problems as omitting necessary content, covering objectives 
incompletely or at an incorrect level, incorrect or inade- 
quate emphasis on particular subjects or concepts, incom- 
plete material supplements, poorly designed teaching 
Strategies, and tests which do not address the objectives. 
Once the workings of the system and the basics of the 


materials are understood, development of the task to obtain 
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FIGURE 3. Design of Problem Solving Task 
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proper responses from the user and the system may begin. 

After this initial phase, the design of the problem- 
solving task should begin with a careful analysis of the 
instructional requirements. The forming of the instrument 
to evaluate a particular concept is obtained through the 
educational process of an instructors preliminary knowledge 
of the subject matter. In this phase, it is important that 
the test has face validity for the objectives which addres- 
ses the particular concept. It is also important that the 
problem-solving task cover the specific concept matter at 
the appropriate difficulty levels. Since this instrument is 
the principal element which will guide the response evalua- 
tion process, it must fully represent the intent of the 
educator to assure adequate performance. Any revisions must 
be reviewed by the educator to insure the technical accuracy 
of the content changes. 

Unless these first two steps are successfully completed, 
all subsequent efforts may be jeopardized. For example, if 
the problem-solving task is so designed that it will not 
evaluate the particular concept, the work in the evaluation 
of principal incorrect responses for that particular concept 
will be largely wasted. 

The final step of the design of the problem-solving task 
includes the determination of the proper responses desired 


and the possible major incorrect responses that may be 
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obtained. Since, the number of possible incorrect responses 
for any particular concept is theoretically infinite, this 

process will only serve as an initial starting point in the 
identification of the principal incorrect forms of that con- 


cept that is taught. 


C. SELECTION OF PROTOCOL METHODOLOGY 

The protocol methodology for the administering and eval- 
uation of the problem-solving task (see figure 4) are given 
as a set of rules. Revision of the protocol may be made at 
any time it is determined that it is non-feasible for a 


particular application. The rules are as follows: 


1. Selection of students for use on the problem-solving task 
should be on a volunteer basis. Using volunteers rather than 
individuals ordered to participate in the problem-solving 


task will avoid some possible attitude problems. 


2. Volunteers should be selected on their qualification for 
the particular concept being tested. They should be se- 
lected to provide a reasonably balanced representation by 


sex, specialization and professional level. 


3. The problem-solving task should be conducted outside the 
classroom and should be informal. Attempting to work within 
the classroom is likely to disrupt regular classroom activi- 


ties. In addition, it has been found that the problem- 
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solving task should be administered to one test Subject at 
atime. Group testing tended to inhibit the subject's will- 


ingness to verbally report their thought processes. 


4. The examiner should be available to answer questions, 
provide guidance where necessary, and note areas where 


students have difficulty. 


5. Finally, and most important, the test subjects are to 
learn the particular concept and apply it to a number of 
practice problems. During the problem-solving task, the 
test subjects are to verbally report their thought proc- 
esses and draw and describe mental images, if any occurred. 
In this way, the subject's thought processes for a partic- 
ular concept may be determined by analyzing the result and 


the sequence of steps that were used to obtain that result. 


The above rules provide effective guidelines for an 
efficient and standard administration and evaluation of 
the problem-solving task. It is noted that verbal re- 
ports of a subject's thought processes may be tape re- 
corded for accuracy and historical references. This is 
only required, however, if it has been found that test 
subjects tend to lean towards verbal reporting vice the 
written method. Continual evaluation of the methodology 
must be made during the problem-solving task for any 


particular situation. 
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De FRETEST 

The fourth component of the procedural model (see 
figure 5) assumes that a dedicated amount of time has al- 
ready been spent in the selection and development of a 
problem-solving test. The purpose of this phase is not so 
much directed at evaluating the response effectiveness of 
the problem-solving task as it is at locating errors of 
content and logic which were overlooked. Typically, the 
number of students tested would not be sufficient to answer 
questions concerning the incorrect concepts that the student 
learns, but do provide an avenue by which the educator can 
learn if the materials are easily understood by their in- 
tended audience. The pretest provide answers to questions 
such as the following. Are there inconsistencies between 
the concept and the problem-solving task? Are directions to 
the student clear enough to be followed? Is wording clear? 
Is the problem-solving task too long to hold the student's 
attention? 

The pretest should generally be administered to 12-15 
Students, but the number will vary with the number of errors 
found early in the pretest. If many errors are found with, 
for example, the first four or five students, it would be 
preferrable to suspend the pretest until the problems have 
been resolved. If available manpower permits, it may be de- 
Sirable to supplement review of the pretest by other instruc- 


tional personnel for the purpose of locating grammatical and 
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typographical errors and problems with clarity. 

The next step pertains to data analysis. With the use 
of imagery, as specified in step 3 of the procedural model, 
principal incorrect forms of the concept that a student 
learns may now be initially determined. These incorrect 
concept forms are to be compared to the theoretical incorrect 
forms established in step 2 of the procedural model. After 
comparison of both the theoretical and diagnosed forms of the 
principal incorrect responses, the educator is now ready to 
begin the next component of the procedural model, modifica- 


tion of the problem solving task. 


E. MODIFICATION OF THE PROBLEM SOLVING TASK 

This component (see figure 6) of the procedural model is 
designed to modify or correct any errors and ambiguities re- 
vealed by the pretest. It‘s purpose is to evaluate the 
instructional effectiveness of the materials to determine 
the specific areas of weakness. Problems which have more 
than one theoretical incorrect response or too many diagnosed 
responses, such that a specific incorrect concept cannot be 
applied to the problem, are to be evaluated and deleted from 
the problem-solving task. If it is found that the problem- 
solving task has too many unresolvable problems, it must be 
determined if the particular subject matter is actually 
testable. If not, the procedural process for a response 
evaluation procedure terminates at this point, otherwise the 


educator is ready to begin the next component of the 
37 
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procedural model, administration of the problem-solving task 


for formative evaluation. 


F. ADMINISTERING OF THE PROBLEM SOLVING TASK 

The sixth component of the procedural model (see figure 
7) assumes that the problem-solving task has now been per- 
fected and is ready to be used to evaluate the incorrect 
responses for the particular concept. A reasonably balanced 
population of test subjects by sex, specialization and pro- 
fessional levels are to be provided. The test is to be 
administered in the same methodology as was used in the pre- 
testing of the instrument. 

The final study should be administered to 20-30 students, 
but the number will vary depending on the particular con- 
cept that is being evaluated. Successful completion of this 
component moves the sequence of the response evaluation 
process to the last component of the procedural model, data 


analysis. 


G. DATA ANALYSIS 

The analysis of the results appears in the final chap- 
ters. However, a brief outline may be useful in interpret- 
ing the data analysis (see figure 8). 

The first step examines the incorrect form of the 
responses by subject and by problem. The influence of abil- 
ity level, sex and type of problem are considered next. Final- 


ly, the data will be analyzed as its applicability to CAI. 
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IV. DATA ANALYSIS 


The final study included sixty-one subjects. Thirty- 
seven of these were obtained from prior research 
(Weissinger~Baylon, 1978) and twenty-four (see Appendix A) 
were obtained during the course of this study. They provide 
a group reasonably balanced by sex, specialization, profes- 
Sional level,and most important, ability to understand the 
particular concept--the upper bound of a set. 

The test group was comprised of 23% females and 77% 
males. Of these, 33% were graduate or undergraduate stu- 
dents in the mathematical sciences and 67% were under- 
graduates or non-mathematical science graduates. All 
subjects were volunteers, and the time required to do this 
experiment varied from 10-29 minutes per subject. The final 
test instrument consisted of a definition, a practice prob- 
lem, and nine problem-solving sets (see figure 9). 

As is true in the analysis and evaluation of any problem- 
solving task, the principal challenge lies in summarizing 
the protocol data. Using data from 671 problems obtained 
during this research, the response evaluation technique for 
the upper bound of a set concept was evaluated from three 
different standpoints: (a) evaluation of its ability to 


isolate the major incorrect responses for a particular 
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FINAL TASK DESCRIPTION 


| Set Number Set 


Definition Let E be a subset of the real numbers, R. 
An element, m, of Ris said to be an upper 
bound of E if x is equal to or less than m 
for every x in E 


Practice Let E be the set Ml, 2, 3743) i.e. the set 
Problem consisting of the first four positive 
integers. Does E have an upper bound? 


The set of natural numbers, N (i.e., the 
positive integers). 


A finite set. 
(lyre... F 
The set of real numbers, R. 


The non-negative numbers 


The closed interval [3,5] 





{ £2,V3,/4,45,16,47, ...} 
{tan x, where x =, 21, 3, 47, 57, 6m, ...} 


(de 224 eri6) li} 





Figure 9. The sequence of steps to perform the task consisted of: 

(1) reading the definition, (2) and then solving the practice problem. 
_ Each subject was then instructed to determine whether or not each of 

the sets had an upper bound. During the process of the problem solving 

task, subjects were asked to provide verbal and written reports of 

their thought processes, and draw mental pictures, if any occurred. 


~ 
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concept, (b) evaluation of response error statistics by 
type of set, and type of set between ability level and sex 
and, (c) evaluation of subjects response pattern. Results 
of each of these three evaluation efforts are reported in 


the following subsections. 


A. ISOLATION OF MAJOR INCORRECT RESPONSES 

Prior research has dictated the existence of an in- 
finite number of theoretical correct and incorrect re- 
sponses for any particular concept, depending on the 
nature of the behavior and the level of skill of the indi- 
vidual. [Taber, Glaser, Schaeffer, 1965, p. 19] . Keeping 
this fact in mind, the isolation of major incorrect 
responses becomes of the utmost importance during the 
final evaluation of any problem-solving task. 

From the educators preliminary knowledge of the con- 
cept material and the initial analysis of the pretest 
results--procedural model, step two and four respectively, 
seven major incorrect responses or incorrect concept forms 
were identified as being possible for the upper bound 
concept. They were classified as follows: 

1. Positive infinity is the upper bound for all 
sets. 

2. Infinite number of points has no upper bound. 

3. An infinite set has no upper bound. 


4. The upper bound has to be a member of the set, 
if the set is to have an upper bound. 
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5. Misreading of the problem. 

6. Misunderstanding the problem. 

7. Other (Ignoring the problem, unable to do 
problem, etc.) 

Analysis of the above incorrect response concepts 
indicated that incorrect concept forms two and three 
(infinite number of points has no upper bound and infinite 
set has no upper bound, respectively) were very similar in 
construct and may be combined to state that: "no upper 
bound exists for an infinite set." In addition, in- 
correct concept form six (misunderstanding the problem) 
May be placed in the incorrect concept form seven (other) 
category, Since the relationship between the two concept 
forms were Similar in nature. This initial detailed 
analysis of all the possible major concept forms resulted 
in the isolation of five incorrect concept forms for the 
particular upper bound concept. 

Application of the five incorrect concept forms to the 
problem-solving sets, illustrated, theoretically, the 
errors that an individual would or should obtain with each 
particular incorrect concept form that he had learned (see 
figure 10). It must be noted that the final test instru- 
ment was designed and modified (procedural model, step 
five) so that no more than one theoretical incorrect 


concept applied to each problem-solving set. 
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Using the above information, the final test results were 
analyzed and the errors were evaluated in regards to the 
five major incorrect concept forms (see figure 11). The 
analysis indicated that 94 percent of the errors fell 
into the particular error category that was determined, 
from the theoretical application of incorrect concept 
forms, to produce an error for the individual problem- 
solving set. In addition, 80 percent of the errors fell 
into just two major incorrect concept categories: plus 
infinity is the upper bound of everything and an in- 
finite set has no upper bound. 

It is reassuring to note that the absence of response 
errors for the major incorrect concept form "the upper 
bound must be an element of the set" indicated that the 
total number of incorrect response forms for this par- 
ticular concept may be further reduced to only four 
major categories. This is a reduction of 43 percent from 
the number initially determined in the early stages of ee 


response evaluation model. 


B. SUBJECT AND SET TYPE EVALUATION 

Results of the "Subject to set evaluation" is discussed 
in two parts: evaluation of type of sets, and evaluation 
of subject, ability level and sex between type of set. 

The errors for the particular sets were tabulated in 


figure 12. The sets are classified as being either 
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"Abstract," those that may require interpretation, for 
example: "The non-negative numbers," and "Concrete," those 
which subjects can analyze directly, for example: 
"{1,2,4,8,16,...}." Statistical analysis indicated that the 
error rates for the abstract sets were considerably higher 
than the concrete sets (72% and 28%, respectively). The 
difference may be attributed to three significant factors: 
the final instrument consisted of five abstract sets com- 
pared to only four concrete sets; the error rate for set 
eighteen is abnormally high; and due to the inherent 


nature of the abstract type of set. 


Deletion of set eighteen from the test instrument to 
remove the problem solving set with the abnormally high 
error rate and to provide an equal distribution of abstract 
and concrete sets, still resulted in an error rate differ- 
ence between the two types of sets of 60 percent for 
abstract sets and 40 percent for concrete sets. Therefore, 
is the inherent nature of the type of sets a Significant 
factor on the type of response? Consider for example set 
eleven: 

Set ll. The non-negative numbers. 
Ten subjects answered it incorrectly: six by incorrectly 
applying the upper bound definition, and four by misunder- 
Standing the term "non-negative." Now, consider a mathe- 


matically similar task, set twenty-one: 
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peel l,2,4,8,16,...}. 

Both sets are infinite with the only difference that set 
eleven is written in abstract terms while set twenty-one 
is written in concrete terms consisting of five elements. 
Since the differences between the sets are minor, error 
rate comparison is Surprising. Indicating that the in- 
herent nature of a problem-solving set is a Significant 
factor in the type of response that may be obtained. 

Next, error response for subject, ability level, sex 
and type of set (see figure 13) is analyzed and provided 


below: 


1. It was found that the overall response error rate for 
the problem-solving instrument was 11.7 percent. Even 
though the test was designed to be easy and administered 
to subjects who had the ability to understand the particu- 
lar concept, this indicates that individuals, as problem 


solvers, are often unreliable for objectively easy tasks. 


2. There waS a Significant difference of 3.3 percent re- 
Sponse error rate between the “mathematical science" group 
and the "other" group. The "mathematical science" group 
consisted of graduates from the Naval Postgraduate School 
and Stanford University and undergraduates from Colorado 
State University in the mathematical sciences (Mathematics, 


Computer Science, Statistics, and Operations Research). 
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SUBJECT IDENTIFIERS, SEX, FIELD, LEVEL, AND ERRORS 





Subject Sex Field MATH YEAR Errors 

Number SCIENCE No. % 
eo. C.D. M CSM GRAD 1 ea 
S2 L.M. F CS X GRAD 2 22 
oo 1.V. M CSM GRAD iL Le 
S4 R.W. M CSM GRAD Zz 22 
ma S.L» M CSM GRAD 0 0 
SO P.H. M cs GRAD 3 33 
Be? =Y.K. M CSM xe GRAD 4 44 
em R.F. M D.P. UND 1 Beal 
mo F.K. M D.P. UND 4 44 
S10 D.M. F MATH Xx UND L 11 
al S$.W. M MoE. UND i da 
S22 B.N. M Ge. UND 3 33 
Sas £.C. F MATH x UND 0 0 
pig G.J. M c3 GRAD 0 0 
BeS> R.C. M Der. UND 1 ae 
S16 K.G. M D.P UND x 11 
Sl7 £.T. M Dir. X UND 2 22 
os J.L. M Cis. GRAD 0 0 
19 L.N. F Cer UND aE Jeu 
S20 T.M. M Dee. X UND 4 44 
eeu A.C. M BoP. GRAD a a 
p22 J.T. M Ces. UND 2 22 
oes J.B. M Mon. UND 0 0 
S24 S.M. M M.E. UND 2 22 
m5 R.E. M Dick. X GRAD 0 0 
fio J.M. M C.c-. Xx GRAD 5 55 
mi7 P.M. M M.D. M.D. 3 33 
T18 L.W. M. MATH xX Ph.D. 0 0 
729 N.B. M MATH x GRAD uk i 
20 R.P. M Coo. X GRAD 0 0 
T21 R.R. M MATH X GRAD 0 0 
ize J.-F. M M.D. x GRAD 0 0 
T23 D.D. M STAT Xx GRAD 0 0 
T24 A.K. M MATH x GRAD 2 el 
Figure 13. The error values are for nine sets. 
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SUBJECT IDENTIFIERS, SEX, FIELD, LEVEL, AND ERRORS 


Math Errors 


Science No. % 





T25 H.S. M MATH x GRAD 0 0 
T26 S.K. M MATH x GRAD 0 0 
T27 B.B. M BIOL UND 0 0 
T28 J.A. F Phys/Cs GRAD 2 22 
2° S.L. F Lang GRAD c z 
T30 P.W. M MATH X Ph.D. 2 22 
T3l1 J.H. M MATH X GRAD 0 0 
®32 A.S. M STAT X GRAD 0 0 
33 Geek. M STAT x GRAD 1 fb 
maa R.G. M PSY UND 0 0 
T35 S.N. M MATH X GRAD. 0 0 
T36 J.N. M C.E. GRAD 3 33 
ms7 J3.X. M ENG AB a! ie 
T38 R.B. M M.E. UND 0 0 
T39 W.P. M M.E. UND 0 O 
M40 D.A. M PSY UND 1 1 
T41 N.A. F EDUC GRAD - = 
m2 J.F. F Cone UND O 0 
T43 B.B. M UNDECL GRAD 0 0 
T44 B.E. M PHYS UND 4 44 
M45 J.R. F UNDECL UND 0 0 
T46 N.Z. F UNDECL . UND a ll 
47 V.L. F UNDECL UND 0 0 
T48 L.H. F UNDECL UND i ll 
T49 C.H. F MATH UND 0 0 
TSO J.H. M CE, UND 0 0 
T51 K.T. F PHYS GRAD 0 0 
752 J.G. F UNDECL UND 0 0 





Figure 13 (con't) The error values are for nine sets. 
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The "other" group consisted of Naval Postgraduate School 
and Stanford University graduates and Colorado State 
University undergraduates in curriculums other than the 
mathematical sciences. The response error rates for the 
groups were: 9.4 percent for the “mathematical sciences," 
and 12.7 percent for the "other" group. This significant 
response error difference provided a strong indication that 
additional years of specialized training does provide an 
improved response performance for technically inclined 


concepts. 


3. The response error rate for males was 12.5 percent, 
for females 8.7 percent. The lower response error rate 
for females 1s very surprising, since today's society is 
Still geared towards the higher technical training of the 
male counterpart. The error rate difference, however, may 
be explained by the unequal test population of females 


compared to males. 


4. As indicated in the set type evaluation, the error re- 
sponse rate for the "Abstract" set category was found to 
be considerably higher than the "Concrete" set category-- 
15.1 percent and 7.4 percent, respectively. This is a 
result of the inherent nature of the problem. It has been 
found that individuals relate better to problems which may 


immediately be visualized. Thereby, reducing the amount 


a 





of time required for the thought process and thus, minimize 


the chance for error. 


5. Response error rates for subject's ability level between 
set type and subject's sex between set type is given in 
figure 14 and figure 15 respectively. The figures indicate 
that mathematical science students are better prepared for 
technical concepts provided in concrete terms, however, 

fail to provide a better score for abstract type problems. 
Moreover, the female is apt to provide better responses than 


their male counterpart for the specific set type categories. 


The above data analysis for the different categories 
indicates that a wide disparity exists between and within 
the groups that were looked at. A split halves reliability 
coefficient, to determine if the nine sets constitute an 
acceptable instrument, was obtained by regressing total 
errors of even numbered subjects on the total for odd 
numbered. The coefficient was r = .66, Suggesting that 


the problem-solving task was an acceptable instrument. 


C. SUBJECTS RESPONSE PATTERN 

Analysis of figures ten and eleven, indicated that a 
definite mastery of an incorrect concept existed for some 
test subjects, while for others it was not so easy to 


determine an error pattern (see figure 16). 


38 








Math Sci. Other 
n = 20 n=41 


Concrete 
sets 


Abstract 
sets 





Figure 14 Response error rates between ability level and 


set type 
Male Female 
n= 47 n = 14 


Concrete 
sets 


Abstract 
sets 





Figure 15 Response error rates between sex and set type 
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SUBJECT RESPONSE ERROR PATTERN 
INCORRECT CONCEPT FORM 


An infinite set has no 
upper bound 





Plus infinity is the 





The upper bound must 
be an element of the 




















No. No. No. 
Theor. Subj. % jTheor. Subj. % 
Errors Errors Firrors Errors 


HANQAAwW ANH 





SZ 
S17 
S20 
S21 
S22 
S24 
T16 
c.7 
T28 
T30 
T33 
T36 
237 
T40 
T44 
T48 





Total | 40 26 


Avg B25 65% 


Figure 16. The number of subject errors are compared to the number 
of problem sets that should have theoretically produced error with 
the particular incorrect concept form. 
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Consider, for example, the major incorrect concept 
form "plus infinity is the upper bound for all sets." 
Subjects S9, T16 and T44 responded incorrectly, using the 
particular incorrect concept, to four out of five problem- 
solving sets which were theoretically designed to produce 
errors if that particular incorrect concept form was 
applied. Their test results definitely indicated a mas- 
tery of the incorrect concept form as illustrated in the 
following S9, T16 and T44 excerpts for set eleven: 

S9: “Since plus infinity 1s the largest possible 
number, it must be the upper bound." 

Zoe pounced by continuum 1.e., infinity of con- 
tinuum greater than infinity of natural 
numbers." 

T44: "Then I remember very strongly that infinity 
is the largest. So u.b. 1s infinity." 

The other missed sets for the particular incorrect re- 
Sponse concept produced a similar type of incorrect reason- 
ing process from the three test subjects. 

Now consider subject T30's response for the same 
incorrect concept form. 130 responded incorrectly to only 
two out of five problem sets. Excerpts from the missed 
problem sets are illustrated below: 

Set 2: "Nothing can be to right of all of then, 

therefore, upper bound exists." 


Set 17: "Square root is unbounded (i.e., off to 
right). Therefore, upper bound exists." 
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The response was implicit, but not stated as clearly. 
Subject T30 seemed to follow the correct process for an 
infinite set being unbounded, but then stated "therefore, 
upper bound exists." 

The analysis indicated that the other errors produced 
by the test subjects followed a similar pattern as was 
shown above. With the exception of S20, no other test 
Subject answered all of the problem sets, designed to pro- 
duce an error for a particular concept, incorrectly. 

It is surprising to note that an error response 
pattern for each subject could not be determined. There- 
fore, is it meaningful to even talk of error patterns? 
Brunner, Goodnow, and Austin (1956) discussed the role of 
errors and indicated that a notable flexibility of the 
subject's behavioral pattern exists. Different 
strategies will be formed for certain information. 
Further explanation for Reece tes in error response 
patterns is provided in the following statement: 

. « « heuristics do not guarantee any solution at 

all; all that can be said for a useful heuristic 

is that it offers solutions which are good enough 

most of the time. 

[Feigenbaum and Feldman, 1963, p. 6] 

This description is consistent with the behavior of 

the subjects error patterns. 


The error pattern for test subjects with the in- 


correct concept form "the upper bound must be an 
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element of the set" were not analyzed, since the test 
instrument was designed to produce no errors for that 
concept--making it difficult to analyze a set error 


pattern. 
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V. CONCLUSIONS AND RECOMMENDATIONS 


The proposed complete response evaluation approach 
described here appears promising. Test results have dem- 
onstrated that the developed technique provides the 
educator with a method to structure the CAI course material 
towards responses which a student is more likely to pro- 
vide and, therefore, will prove to be most effective for 
the individual student. As it stands, the approach is 
ready for use by CAI lesson writers and evaluation per- 
sonnel. This final section summarizes the observed 
strengths and weaknesses of the proposed technique, 
Suggests specific areas for further development, and out- 
lines a number of recommendations for utilizing the 
response evaluation approach. 

A. EFFECTIVENESS OF THE RESPONSE EVALUATION PROCESS AND 

SUGGESTIONS FOR FURTHER DEVELOPMENT 

On the basis of test results discussed in the preceding 
sections of this report, it can be concluded that the re- 
sponse evaluation process for a particular concept was 
reasonably effective; substantially more than the current 
processes that it may replace. Use of this approach re- 
Sulted in the dramatic reduction of the infinite number of 
possible theoretical incorrect responses for a particular 


concept to a mere three. Also, subStantive, although not 
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striking, improvements in the methodology for administering 
the final instrument were discovered. 

It must be admitted that, in terms of one of the more 
central measures of success, discovery of an response error 
pattern for each individual subject, is not as dramatic as 
Might have been hoped for. A set error pattern was dis- 
covered for some individuals, but could not be generalized 
for all test subjects. 

Two major processes of the response evaluation approach, 
the methodology used, and the error response pattern will 
now be examined and discussed in some detail. 

1. Methodology Used 

The imagery and verbal response methodology is sound. 
Although there is no proof that imagery dramatically reduces 
problem-solving errors (Weissinger-Baylon, 1978), it is an 
effective method for evaluation of thought processes for the 
particular application for which it was used. It must be 
noted, however, that some problem-solving tasks make heavy 
demands on imagery, while others do not. This is not the 
reSult of the inability of the subjects to produce them, 
but on the inherent nature of the task. While problems 
were encountered in two areas, they were more of the nature 


of management problems rather than problems with the metho- 


dology per se. 

First, the test subjects were unaware of the process of 
uSing imagery in their problem-solving task and were re- 
luctant and not ready to accept its use. Subjects had to 


be continually urged to put down any mental images, if and 
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when they occurred. This reluctance to describe mental 
images is not uncommon in the imagery controversy, which 
dates back to the early part of this century. The imagery 
controversy raged between psychologists with Holt (1964) 
and Titchener (1926) pleading for a new and more direct 
consideration for mental imagery, while behaviorists re- 
jected the idea as unscientific. It was not until the mid- 
Sixties when Shepard (1978) and others discussed the im- 
portance of imagery in the problem-solving role, that it 
was more readily accepted by cognitive psychologists. 
Therefore, in addition to the recognized newness of the use 
of imagery in problem-solving, the further reluctance of 
test subjects to use mental images to solve a problem- 
solving task may best be explained by Arnheim: 
images must be highly abstract since the mind operates 
often at high levels of abstraction. But to get at 
these images is not easy. I mentioned that a good deal 
of imagery may occur below the level of consciousness 
and that even if conscious, such imagery may not be 
noticed readily by persons unaccustomed to the awkward 
business of self-observation. At best, mental images 
are hard to describe and easily disturbed. 
(Arnheim, 1969, p. 116] 
There is no doubt that the use of mental imagery in 
problem-solving can be, and hopefully will be refined 
through subsequent psychological studies. Once the sub- 
jects are aware of the role of mental imagery, their 


reluctance to report it will diminish and its management 


wlll become more efficient. 
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The second area concerned over-emphasizing verbal re- 
porting versus the written reports. Since it is easier to 
speak than to write, some subjects initially tended to 
"overplay" their verbal responses, while "down-playing" the 
written protocol. Even though more accurate and complete, 
analysis and management of the taped verbal protocols 
became difficult when it could not be compared to the 
subject's written protocols. Further, to consider the 
opposite extreme, some subjects tended to shy away from 
any verbal responses when they discovered that they were 
to be tape recorded. Therefore, even though tape record- 
ings are more accurate and complete and, in some cases, 
also provide historical reference, their value is depend- 
ent on its application and its economic feasibility. 

2. Error Response Pattern 

One of the main goals of this study was to discover a 
response error pattern for each subject. This discovery 
would have led to a more individualized package of instruc- 
tion for the intended user. However the results were less 
than dramatic, showing irregularities and inconsistencies 
in subject responses. For example, even though test re- 
sults for some subjects definitely indicated that they 
had learned an incorrect form of the concept, obtaining 
an error rate as high as four out of five possible errors, 
it was not conclusive. An important question that could 
be asked from these results: "if the incorrect response 
form was definitely learned, why were not five out of five 


errors produced?". Another example, showing irregularities 
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or "funny stuff" in subject's responses are shown in test 
results where the subject is successfully progressing 
through the problem, but then supplies the wrong response. 
The principal challenge, therefore, lies in analyzing the 
protocol data and answering the following questions: 


(1) How may these inconsistencies and irregulari- 
ties be explained? 


(2) Where do we make the cut-off mark to deter- 
mine if a subject has learned an incorrect 
response form: is it at fifty percent 
incorrect responses for a particular con- 
cept or dare we raise or lower this mark? 

These are difficult questions. A possible solution to 
the first question may best be explained by Bruner, 
Goodnow and Austin: 

- . . Strategies as employed by people are not fixed 
things. They alter with the nature of the concept 
being sought, with the kinds of pressures that exists 
in the situation, with the consequences of behavior, 
etc. And this is of the essence. For what is most 
Creative about concept-attainment behavior is that the 
patterning of decisions does indeed reflect the de- 
mands of the situations in which the person finds 
himself. 

(Bruner, Goodnow, Austin, 1956, p. 55] 

In addition, Feigenbaum (1963) pointed out that some- 
times the process we use to discover or reveal something, 
produces incorrect responses. The interesting question 1s 
"how often"? 

There have been many theories on how the human mind 


processes a problem-solving task. One of these, which 


seemed to be the most plausible, and from the protocol 
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data, the one generally followed in this study, stated that 
the problem-solving process depended upon the presence of 
certain previously learned rules (or in simpler cases, 
concepts). These previously learned rules and concepts 

had to be recalled, as a first step, using instructions, or 
in the case of this experiment--the upper bound definition. 
Whatever stimuli was used to recall the previously learned 
rules, the externally applied guidance presumably functions 
to increase the vividness (or "“availability") of some 
previously learned entity. 

At the next stage, the individual searched for and 
selected the recalled rules that were specifically relevant 
to the stimulus situation, and rejected those which were 
not. There is bound to be some "noise" at this stage of 
the game, and the process is one of distinguishing the 
noise from the relevant signals. For example, if the sub- 
ject is searching for the upper bound of a set, the 
geometric progression of the set may be quite irrelevant. 
If he is searching for the trigonometric relationship for 
"tangent" at different values for "pi," the size of the 
triangle may be quite irrelevant. Conceptualization of 
these features of the stimulus situation may have been re- 
called; but they must be distinguished from other concepts 
and discarded. 


Combining subordinate rules was the next stage of the 


problem solving process. There are many ways in which 
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recalled rules and concepts may be combined; the object is 


to find the correct one. For example, consider set five: 


seta: 1 (ly 1/2, 173,.-..} 


The three dots (...) would indicate the rule that the set 
is infinite. However, the set also stresses consideration 
for the rule when the numbers are progressing in descending 
order. Presumably, in this stage, the rules were narrowed 
to a relatively few likely combinations, and thus reduced 
the time of search. 

At the next stage, the subject arrived at a provisional 
rule which he believed may solve the problem. This rule 
was then subjected to verification by carrying out the 
operations it suggested with reference to the stimulus 
objects. 

The final stage that the subject was found to perform 
was verification. Here, the provisional rules were 
checked by application to the specific problem. Should 
verification not work, the subject returned to the task of 
trying new combinations of rules. When verification did 
work, the solution had been acquired. 

In summary, the problem solving process used may be 
thought of as a linear sequence of operations that are 
actively carried out by the subjects according to a system 
of links influenced by a set of rules and laws. The in- 


consistencies and irregularities of the subject's responses 
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may be a result of the breaking of these links or wrong 
applications of the rules. Mackworth (1965) compares the 
human mind to a very efficient computer, composed of a 
large number of programs to handle all required situations 
(with modifications). He states that the principal 
Criteria of problem solving is “choosing correctly between 
existing programs." Incorrect selection of these programs 
will provide incorrect or inconsistent solutions to the 
problem~solving task. 

The answer to the second question must be made on a 
judgemental decision, taking into consideration the 
objectives of the particular concept. The analysis has 
indicated, however, that if there was less than fifty per- 
cent error response rate for a particular response error 
form, a definite error pattern could not be determined. 

In conclusion, the many unanswered questions for de- 
fining a specific error response pattern must be left for 
the behavioral psychologists. In this study, the attempt 
was made to discover a specific error pattern for a par- 
ticular concept, using mental imagery as a tool. A few 
explanations were provided for the inconsistencies and 
irregularities for subject's responses, but much remains to 


be done in this area of response evaluation. 
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B. RECOMMENDATIONS FOR THE RESPONSE EVALUATION APPROACH 
IN THE CAI ENVIRONMENT 


Both Abrahms (1971) and a report on “materials develop- 
ment procedures and evaluation," by the McDonnell Douglas 
Astronautics Company (1980), indicated that a major short- 
coming in CAI development is in the area of response 
evaluation. They stressed a need for a method which would 
guide the lesson writer, by defining more sophisticated 
rules by which anticipated responses to constructed re- 
Sponses may be judged. The technique provided in this 
study seems to fulfill some, if not all, of these require- 
ments. 

The major conclusions drawn from this study pertain to 
the finding that the response evaluation approach can be 
used to develop CAI lessons with the individual student in 
mind rather than to provide an additional step in the 
already tedious writing of course materials per se. Thus, 
the intent of the recommendations outlined below 1s to 
Suggest potential benefits to be derived from broader 
application of an instructional technology which has been 
shown to be reasonably effective in this context. 

An obvious next step would involve further development 
and implementation of the response evaluation approach in 
the actual CAI course material development process. The 
area which appears particularly promising is the identifi- 


cation of the principal incorrect responses for a particular 
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concept. The educator may now define his lesson branching 
structure with these incorrect response forms in mind. 
Providing such system responses as: 
(1) “Are you aware that plus infinity is not the 
upper bound for all sets," 


(2) “Are you aware that an infinite set may have 
an upper bound," 


(3) and finally, "Are you aware that the upper 
bound for a set does not necessarily have to 
be a member of the set." 

With some modification and further study, the response 
error pattern process could be implemented to provide for a 
more totally structured individualized CAI system for a 
particular concept. 

Eventually, one of the most promising areas for the 
response evaluation approach would be to broaden the area 
of CAI instruction to a more diversified field of 
knowledge. For example, CAI lessons could be prepared for 
fields which were generally considered too complex due to 
their psychological response processes for their mastery. 

Finally, it is recommended that any implementation of 
the response evaluation process be coupled with the follow- 
ing guidelines, proposed by Colonel Joe Nastasi, if the 
CAI lesson is to be a success: 

(1) CAI lesson writers must be senior instructors 


who are experienced in their subject matter 
areas and in the course to be developed. 


is 





(2) 


(3) 


(4) 


(5) 


(6) 


7) 


(8) 


CAI lesson writers should be trained programmed 
instruction writers. 


Learning how to write viable and valid lessons 
takes considerable time, and constant turnover 
of authors is disastrous to satisfactory 
progress. 


CAI lesson writers must be assigned full time 
duty of producing lessons for the CAI program. 


Close and continuous liaison must be main- 
tained between the course, other instructors, 
CAI lesson writers, the educational con- 
sultant and the CAI project operations. 


An authorized course subject block diagram, 
with indicated lessons, examination questions, 
behavioral objectives, plan of each lesson, 
and lesson flowcharts must be prepared prior 
to writing and coding lessons, and be avail- 
able to insure valid CAI lesson development. 


The officer in charge and chief instructor of 
the course being developed must retain full 
control and supervision of the CAI lesson 
writers and their efforts. 


All CAI lessons must be very carefully re- 
viewed from a pedagogical and student view- 
point. 


[Prokop, T9767.) . 218-219] 
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APPENDIX A 


TEST DATA FROM SUBJECTS 
Sl CDG 


ear C.D. 
DEFINITION: Let E be a subset of the real numbers, R. An element, 


m, Of Ris said to be an upper bound of E if x is equal to or less than 
m for every x in E. 


iG R 

2. mER 

3. ~x =m— nm = upper bound. 

PRACTICE PROBLEM: Let E be the set fl, 2, 3, 4}, i.e. the set consisting 
of the first four positive integers. Does E have an upper bound? 

1. See four finite elements. 


2. Looked for greatest value. 


3. Yes, has upper bound of 4. 
SET 2: The set of natural numbers, N (i.e. the positive integers) 


1. I see an infinite set. 
{| __» 
2. An infinite set is unbounded. 


Se NO. 
SET 4; A finite set. 
1. No mental image at first. Thought about it. 


Then saw 100 tanks. 


2.3 100 
oe 


@ ere @ 
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SC... 
2. Yes, there is upper bound of greatest element in set. 
SET 5: et 2p 7 oy, S08 
1. I see infinitely smaller numbers. 
2. Therefore upper bound must be the greatest number in the set. 


SET 6: The set of real numbers, R. 


1. No. 


2. An infinite set (=)unbounded). 
SET ll: The non-negative numbers. 


1. Infinite set starting at Q. 





? 


2. No, same as set 6. 
SET 13: The closed interval [ 3, oul 


1. Yes. 


o-——_—__—_—_—-» 


2. The largest element in set is upper bound. 


3. Finite set. 
SET 17: {v2, V3, “4, V5, “e, 7, aS 


l. Yes. 


V2 ———> 








Ser. 


2. after test 


7 


~~ 
SET 18: § tan x, where x =7, 2 37, 44, 5/4, 67, 7. 


No. 


1. Yes. (guess since tangent is bounded by 0 , l). 


2. 1 see an set of Q's and l's 
$O.1.0.14 -.-} 


SET 21; a, ea 3. 16, oor 


1. I see numbers increasing. 


2. No, infinite set. 
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S2 LL.M. 
S2 LL.M. 
DEFINITION: Let E be a subset of the real numbers, R. An element, 


m, Of Ris said to be an upper bound of E if x is equal to or less than 
m for every x in E. 


1. Definition thought: 


The real numbers ordered on a number line. 


wees tO 1 2. 3 
7 


PRACTICE PROBLEM: Let E be the set Sa, Zens 42, i.e. the set consisting 
of the first positive integers. Does E have an upper bound? 


i 12 3 4 
2. upper bound = 4. 
SET 2; The set of natural numbers, N (i.e. the positive integers) 


i ee Sais cls 


2. No bound, it goes to infinity. 


SET 4: A finite set. 





1. Points on the line -- bounded by the element which falls farthest 
to the right on the number line (like definition). 


SET 5: 2 aby 2p ap os 
1. I see an infinite number of points getting smaller. 
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s2 o.M. 


W735 2 
3. bounded by 0. 


4. I remember definition and it asks for upper bound, therefore=> 1. 


SET 6; The set of real numbers, R. 


1. no bound. 


2.0 > 





SET ll: The non-negative numbers. 


1. I see an infinite set. 
2. Upper bound & (the first non-negative number). 


le eee 72 wl O 
<——____}+—__+_ ++ 

4. [No lower bound ] 

SET 13: The closed interval [3, ea 


l. Saw the largest number 5. 
Bis 3 5 
3. bound 5. 


SET 17: Sv2, V3, V4, V5, Ve, V7, sey 


1. No bound -- each number bigger than the previous number. 
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SET 18: $ Tan x, where x = 7,27,37,47,57,,67, 1222. 


1. I could not remember definition of tan. 
2. I see ... , therefore goes to infinity. 


3. No bound. 
SET 21: C1, pe ye ahap vee de 
1. Increasing infinite set. 


2 cue 


2. no bound. 
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Soon. V . 


Som. V. 


DEFINITION: Let E be a subset of the real numbers, R. An element, 
m, Of Ris said to be an upper bound of E if x is equal to or less than 
m for every x in E. 


1. x2 m for every x in E. 


PRACTICE PROBLEM: Let E be the set $1, 2 3 4? , 1.@e. the set consisting 
of the first four positive integers. Does E have an upper bound? 


l. Yes, 4. 


2. Four is the largest number and so all of the others are less than 
or equal to it so it must be the upper bound. 


SET 2: The set of natural numbers, N (i.e. the positive integers). 


1. No upper bound —~pinfinite. 


2. Imagery-- (1, 2, 3, ...) 
SET 4: A finite set. 


1. I see a certain number of points. 
2. Imagery -- m1, 2, 3, 4 3 


3. Upper bound equals largest number in set. 


SET 5: £1, v2. ie 3 


1. 1 is upper bound. 
2. I visualize the numbers trailing off to right so 1 is largest. 


My first impression was that it was an infinite set with no 
upper bound. 
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Sol. « 


SET 6: The set of real numbers, R. 


i. 


aie 


1. 


Ze 


We 


2. 


Infinite set, no upper bound. 


Imagery wile 2, 3, % sed. 


SET 11: The non-negative numbers. 


Infinite set. 


no upper bound (see above). 


SET 13: The closed interval i 5 |. 


5 is the upper bound. 


It is the largest number in a finite set. 


SET 17: § V2,V3,V4,V5 V6 ,V7, a } 


2. 


2. 


ore 


Infinite set -- no upper bound. 
I first look at the values of the perfect squares. 


Imagery 


vi “4 Oa 
}—______¢-____-+--__> 
1 2 3 


SET 18: £ Tan x, where x = 7, 27, 37, 47, 57, 67, nee 


i. 


ae 


Visualize coordinate plot of trig values. 


tan = sin 


iis 





3. Infinite set -- no upper bounds. 
SET 21: C1, 2, 4, 8, 16, --} 


1. Infinite set no upper bound. 


2. Imagery = xe 
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S4 R.W. 


S4 R.W. 
DEFINITION: Let E be a subset of the real numbers, R. An element, 


m, Of Ris said to be an upper bound of E if x is equal to or less than 
m for every x in E. 


1. DEF: For all answers oo is no bound 


PRACTICE PROBLEM: Let E be the set f1, Zao a, i.e. the set consisting 
of the first four positive integers. Does E have an upper bound? 


1. E has an upper bound. 
2. m= 4, because set E has ascending order. 
Sle 1 2 3 4 

}-—_}-——+_—+} 


SET 2: The set of natural numbers, N (i.e. the positive integers). 


jet, 2.3 4 4... +00 
}——}—_—_+_+-— 


2. There is no upper bound, unlimited. 
SET 4: A finite set. 


1. A finite set not necessarily has an upper bound, because "finite" 
is concerned with the number of elements in the set. 


2. After asking examiner -- this set is bounded above. 


Beimsees b,"1/2, 173, ... } 


ab 1 0 
pt Ht 


2. The upper bound of this set is 1, because the set has a descending 
order in its elements. 
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S4 R.W. 


SET 6; The set of real numbers, R. 


a, oOo - 0 + cO 
—————EEE——————————— 


2. The set of real numbers has no upper bound because they go to + 
SET ll: The non-negative numbers. 


ie. ee 
0 


2. No upper limit, goes to cod 


SET 13: The closed interval (3, 5 |. 


3, 5 | 
2. The closed interval has an upper bound. 


3. m= 5. 


SET 17; § V2,V3,V4,¥5 V6,V7, wee} 


I eV VM ow. 
—+—+- + 


2. no upper bound 


SET 18: $ Tan x, where x =77, 27, 377, 477 57. 677 ks 








2. Set contains all zeroes, ESo, ee ef 


3. Set has no upper bound. 
SET 21: £1, 2, 4, 8, 16, eas 


1. Set has no upper bound. 


2. Set of the squares of natural numbers 
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S4 R.W. 





S5<S.h. 
o> S.L. 
DEFINITION: Let E be a subset of the real numbers, R. An element, m 


of R is said to be an upper bound of E if x is equal to or less than 
m for every x in E. 


. /() 


R 
Des m2Z[E] 
a. x =[E]< m 


4. m 


PRACTICE PROBLEM: Let E bethe set {1, 2, 3, 4}, i.e. the set consist- 
ing of the first four positive integers. Does E have an upper bound? 


ie E| m2. 3, 4) 


SET 2: The set of natural numbers, N (i.e. the positive integers). 


1. Natural numbers, N, —+>unlimited positive number. 


2. So there should not be any upper bound. 
SET 4: A finite set. 


l. A finite set weenie set is finite. 


2. So should be an upper bound of the subset E. 


fer 5: { 1, 1/25.1/3, «..} 
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Sono .i. 


ety 2, 1/3, ..+) 2 unlimited. 
fee raceion_~ 1/n. 


3. 1 = the upper bound. 


SET 6: The set of real numbers, R. 


1. Undefined property — > 


2. The set of real numbers, R,—»if it means all real numbers —_5then 
there is no upper bound. 


3. If it starts from a certain number downward__> there is an upper 
limit e 


SET 11: The non-negative numbers. 


1. All non-negative numbers. 


2. No upper bound except 


SET 13: The closed interval [3, 5 |. 


lL. +4 
3 5 


2. 5 = upper bound. 


SET 17: $ ¥2,v3,V4,V5,V6,V7, ... 5 


1. (43, ....V7, ...) => Va 


naar ad 


2. —— >> no upper bound. 


SET 18: {Tan x, where x =//, Dir 37, 47, S77, 67, Bea 
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Tan 7 = Tan > Tm 
2. upper bound = .0 
SET 21: aly Dea aia, a 


1. See an infinite increasing set. 


2. no upper bound => 2™, 
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SG P.u. 


eorP.H. 


DEFINITION: Let E be a subset of the real numbers, R. An element, 
m, Of Ris said to be an upper bound of E if x is equal to or less than 
m for every x in E. 


1. m€& R,. 


PRACTICE PROBLEM: Let E be the set $1, iS a} , 1.e. the set consisting 
of the first four positive integers. Does E have an upper bound? 


l. Yes 


2. upper bound = 4. 

SET 2: The set of natural numbers, N (i.e. the positive integers). 
iets 51,92, 3, ...} 

2. Yes, —>always a real number that greater or equal to the largest 


integer. 


SET 4: A finite set. 





A A =$ al, Gay sess ai} 
where iis finite number 


2. yes bounded. 


gars: §1, 1/2961/3, oa 


l. The set decreases. 


2. Yes, bounded = 1. 
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SO P.H. 


SET 6: The set of real numbers, R. 


1 ¢—_-> 


2. No, real numbers not bounded upper side. 
SET 11: The non-negative numbers. 


1. No 


2. Non=-negative numbers could be real — > so the same as set 6 above. 
SET 13: The closed interval 3; 5 | 
di e) = 
-——————-1 
2. Yes, bound =2 5. 


SET 17: § ¥2,V3,V4,V5 V6 V7, itt 


1. Yes, because the number under square root are integers; we always 
could find a real number that is larger than that. 


2. V2 V3 VE ww. OVA a 
== = _ [= = ee 


SET 18: $ Tan x, where x =7, 27, 37, 47, 57, 67, ea 


L. > 


2. always tan x = 0 


3. yes, an upper bound 


Si 





SET 21: 32, De ape 16, vf 


Set of integers. 
Yes would be upper bound. 


Same as set 2. 
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Sievok. 


S7 mK. 


DEFINITION: Let E be a subset of the real number, R. An element, 
m, Of Ris said to be an upper bound of E if x is equal to or less than 
m for every x in E. 


PRACTICE PROBLEM: Let E be the set $1, 2, 3, 44, i.e. the set consisting 
of the first four positive integers. Does E have an upper bound? 


1. Yes, the upper bound is 4. 


2. Each number in the set is less than 4. 
SET 2: The set of natural numbers, N (i.e. the positive integers). 


1. Yes 
2. There is a number greater among the other numbers. 
3. Examiner questions answer: 


4. Infinity is the bound. 
SET 4: A finite set. 

l. Yes, same as set 2. 
em 5 1, 2/2, 1/3, soe f 


l. yes 
2. The upper bound is l 


3. Each number in the set is less than l. 


SET 6: The set of real numbers, R. 





l. Yes, same as set 2. 
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SJ YCKs 


SET 


ll: The non-negative numbers. 
Yes, same as set 2. 


13: The closed interval |3, 5|. 


SET 


Yes. 


5 is the upper bound. 


SET 


i: 5 V2.,V3,V4.V5,VON7, ne 


No, because it is infinite. 


SET 


18: } Tan x, where x =7/, 27, 37, 47, 57, 67, ot 


No upper bound 


Goes to infinity 


21:§1, 2, 4, 8, 16, soe) 


No because it is infinite. 
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SeeR.E. 


S8 R.F. 
DEFINITION: Let E be a subset of the real numbers, R. An element, 


m, of Ris said to be an upper bound of E if x is equal to or less than 
m for every x in E. 


a § Rr} E 
Dr fe-- mf] 
subset E of real numbers, R. 


saan x = Mm 
4. mis only an upper bound of subset Eif x =m 


R 


PRACTICE PROBLEM: Let E be the set jl, 2, 3, at, i.e. the set consisting 
of the first four positive integers.” Does E have an upper bound? 


1. E has an upper bound (4) 


2. Reason is that it is a finite set, so the highest number is the upper 
bound. 


SET 2: The set of natural numbers, N (i.e. the positive integers) 


1. $1, piece n } 


2. infinite set 





3. no upper bound 
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SET 


Souk’. 


4: A £inite set 


i fu, pes. 4. 5 ¢ 


2. 


has upper bound -- in my ex. upper bound is 5. 


SET 5: 1, 1/2, 1/3, 00 


ie 


Zs 


a 


SET 


ee 


decreasing set to infinity. 
def. of upper bound means the highest number in that set. 


The upper bound, in ex., is the highest number (1). 


6: The set of real numbers, R. 


Does not specify what R contains. 

It is an infinite set. 

Correction -- it does specify what R contains -- all real numbers. 
All real numbers is an infinite set. 


Thus no upper bound. 


SET 


ll: The non-negative numbers. 


Means all positive numbers. 
All positive numbers is an infinite set. 


No upper bound. 


SET 


13: The closed interval [3. 5 | 


Contains only numbers 3 and 5. 
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SET 


SS R.2. 


Finite set 


Upper bound is 5. 


17: §V2V3.N4V5 NE V7, eae, 


Increasing set. 

Increasing by one square root digit. 

In order, so that the next number will be 
v8, V9, Vio, ... MN} 

It is an infinite set. 


Does not have an upper bound, except for the Vn, which is undefined 
at present. 


6. I would say this set does not have an upper bound. 
SET 18: [ Tan x, where x =7, 27, 37, 47, 57, 67, weed 
l. Increasing set. 

2. Same as problem set 17. 

3. No upper bound. 

ser 21: {1, 2, 4, 8, 16, ...} 

1. increasing set in sequence Ofeor 

2. infinite set. 

3. No upper bound. 
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Sah eK. 


sa @.K. 


DEFINITION: Let E be a subset of the real numbers, R. An element, 
m, Of Ris said to be an upper bound of E if x is equal to or less than 
m for every x in E. 





4. xizn. 


PRACTICE PROBLEM: Let E be the set {1, Ze 3, 4,$ , 1.e. the set consisting 
of the first four positive integers. Does E have an upper bound? 
1. I see a finite set. 


2 1 2 3 4 
eo. 14s» 


3. According to definition xi= m 


4. Yes, upper bound = 4 or greater. 
SET 2: The set of natural numbers, N (i.e. the positive integers). 


1. what is a natural number -- is 0 a natural number? 


2. This set goes to infinity 


? 20 
—— ++ —-——_—__ 9 
Oe ee ee 
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35 


SIar. i. 


Since infinity is the largest number -- the upper bound is 
infinity. 


SET 4: A finite set. 


le 


a 


Se 


afi, 2, 3, 4, 54 


This is just like the practice problem. 


Upper bound. 


SET 5: {1, 1/2, 1/3, yak 


le 


By 


4. 


I see the set decreasing. 
1/4, 1/5, 1/6 —=>» 1l/n 
Since it is decreasing the largest number must be the upper bound. 


Yes, upper bound = l. 


SET 6: The set of real numbers, R. 


ee 


what is a real number? 

Rational numbers such as -3/4, 18/29, 3, -6 
Irrational numbers such as V2 \V'5, \7 -3 
It (real) goes to infinity. 


Infinity must be the upper bound, since there is no number greater 
than infinity. 


SET 11: The non-negative numbers. 


i 


I see a line going towards the left to -© 





So Boks 


2. Therefore zero must be the upper bound. 
3. Wait, it wants non-negative numbers. 
4. The line goes to the right to +o 
5. Since + tis the largest possible number, it must be the upper 
bound. 
SET 13: The closed interval Ps 5 | 
1. I see a line like this 
aa 
2. From previous experience this is a finite set, with boundaries 
3 and 5 
3. upper bound of 5. 
SET 17: 3V2 V3 ,V4,V5 Ve V7, Bey 
1. I first see an infinite set. 
2. I then look at the "V " 
3. I ignore this, since infinity suggests an upper bound. 
4. Yes, upper bounded above. 


SET 18: } Tan x, wherex =7/, 27, 3% 47, 517, 67, ee. 


aoe 


ae 


3. 


I see an infinite set. 
I immediately say, yes bounded above. 
I look at what tangent means 

sin 


tan = --— 
cos 
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so f.K. 


4. what are the values for tan at 7, 277, etc. 


5. I draw sine curve and cosine curve. 


een 


6. Determined that tan x = $0,0,0,... $ 
7. Upper bound of O or greater. 

SET 21: jue Dy 4, Gy 16, sf 

l. I see a progression -- 2* 


2. It goes on -- no upper bound. 
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S10 D.M. 
eo D.M. 
DEFINITION: Let E be a subset of the real numbers, R. An element, 


m, Of Ris said to be an upper bound of E if x is equal to or less than 
m for every x in E. 


i. U 


2a x =mZ=7R 


PRACTICE PROBLEM: Let E be the set i, C3; 43 , 1.e. the set consisting 
of the first four positive integers. Does E have an upper bound? 


1. This problem is obvious. 


24 123 4 
3. it is finite, therefore upper bound of 4. 
SET 2: The set of natural numbers, N (i.e. the positive integers) 


1. I immediately see an infinite set going towards plus infinity. 


+c 
———___ —_—__——__ > 


2. This implies to me that no upper bound exists. 
SET 4: A finite set. 


1. Obvious, no upper bound. 


SET 5: ule weiss... i 
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SlOra. MM. 





1. I see an infinite set. 
2. It decreases to zero —ptherefore, the largest number must he the 
upper bound. 
3. Almost tricked me by showing infinite set. 
SET 6: The set of real numbers, R. 
1. I immediately see an infinite set, which causes me to say no upper 
bound. 
2. I see no other images. 
SET ll: The non-negative numbers. 
l. I draw a picture 
ee -© 
= 0. 
2. Since 0 is the first non-negative number, it must be the upper bound. 
SET 13: The closed interval [ 3, s | 
1. There are an infinite number of points between 3 and 5, but endpoints 
eaGemouand 5. 
3 5 
if 
i—_ 
2. Yes, upper bound. 
SET 17: {v2 V3, VEN5 NENT, - 
1. I see an infinite set. 
2. I ignore the uw 


IOs 





SLOT .M ; 


3. Set goes to infinity 


4. No upper bound 
SET 18: { Tan x, where x =7, 27, 37, 47, 577, 6/7, ¢ 


l. Tan x = x/y 


8 &D aD CB. 


3. whenever x = 0, y is either l or -l 
4. therefore tan x =fo, 0, O, wef 


5. Set has an upper bound 
SET 21. = 2, 4, 8, 16, +} 


l. I see an infinite set. 


2. Progression = 2a 
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Sl1 S.W. 


eu. S.W. 


DEFINITION: Let E be a subset of the real numbers, R. An element, 
m, Of Ris said to be an upper bound of E if x is equal to or less than 
m for every x in E. 


i. Xz E_5R 
Zs 

R 
See mM sR 


4. x=m 


PRACTICE PROBLEM: Let E be the set { 1, Zt a} , 1.e. the set consisting 
of the first four positive integers. Does E have an upper bound? 


ds R 


2. Since E is a finite set, there exists a R greater than E. 


3. Upper bound. 


SET 2: The set of natural numbers, N (i.e. the positive integers). 





1. No real images. 


2. Since natural numbers go on indefinitily, it implies an infinite 
set. 


3. No upper bound. 
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SET 4: A finite set. 


1. How can I assign an m? 


2. I don't see any numbers or elements. 


SET 5: f1, Wee 1/3. an 


1. I see 
2. Infinite set has no bound, but wait it is decreasing. 


3. I draw picture 


<—+—_____+—_——__| 
1/3 172 i 


4, The progression is 1/n _s goes to 0 


5. Upper bound is the largest number -- 1. 


SET 6: The set of real numbers, R. 





1. I see this immediately as an infinite set. 
Therefore not bounded. 


2. No mental pictures appeared. 


SET 11: The non-negative numbers. 


1. Draw picture of the non-negative numbers. 


——__—____——_—-_>? 
0 + 0° 


2. Goes to infinity. 


3. No upper bound. 
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Sll S.W. 


SET 


SET 


13: The closed interval | 3, 5| 


First looked at bracketed numbers and recognized m=5. 
Next looked at the word interval this implied infinite set. 


Since it is bracketed m2 5. 


17: $\/2,V3,V4M5 Ve M7, ees g 


Saw numbers with square root signs. 
Saw dots which implied infinity and no way of assigning an m. 
numbers are increasing under square root signs. 


infinite set, m can not be defined. 


SET 


SET 


18:4 Tan x, where x =77, 2%, 37, 47, 57, 67, ert 


First saw an infinite set. 
The value 7% for tan looks familiar. 


Draw picture 


tan x= sin x 5» 7 =0 27 =0 
cos x 


the set is {0, Or eG 0 <2. \. 


upper bound. 


Dae {1, 2, 4, 8, 16, —. 


infinite set came to mind first. 
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Sll S.W. 





Same as set 17. 





S12 BN. 


S12 B.N. 
DEFINITION: Let E be a subset of the real numbers, R. An element, 


m, Of Ris said to be an upper bound of E if x is equal to or less than 
m for every x in E. 


l. First time I read definition it didn't make sense. 


2. EER 
3. mER 
4. x€eE 


5. If all of the above true, then 
x=m 

for upper bound to be true. 
6. Checking to see if that makes sense. 
PRACTICE PROBLEM: Let E be the set f1, 2, 3, 4,$, i.e. the set consisting 
of the first four positive integers. Does E have an upper bound? 
l. Instantaneous reaction. 
2. Verify upper bound, also lower bound. 


3. Yes, finite and bounded in both direction 
SET 2: The set of natural numbers, N (i.e. the positive integers). 


1. I see an infinite line going to the right. 


—_—————— 


2. i remember definition said real numbers 


3. Since infinity of real numbers not same as infinity of natural 
numbers. 
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Siz BN. 


4. The set is bounded by infinity of real numbers. 


SET 4: A finite set. 





1. It has bound (from memory, practice problem). 


SET 5: { neeny2 2/32 se 


1. see infinite set. 

2. inspect 

3. Find pattern — >l/n. 

4. Get smaller to right. 

a a: Find largest number. 


&. Bounded by 1. 
SET 6: The set of real numbers, R. 


1. This is just like set 2. 
2. Is there a number = infinity for real. 


3. No,se bounded by infinity. 
SET ll: The non-negative numbers. 


1. These are natural numbers. 
2. This type of question was already asked. 
3. Is this a trick?? 


4. It goes to infinity,’, bounded by infinity of real. 


110 





Siz S.N. 


13: The closed interval [3, 5] 


SEL 


lig 


Zz. 


ae 


SET 


I see 3 and 5. 
5 «ts largest. 


upper bound. 


17: §V2 V3, MiV5 VoV/7, eee; 


This is an infinite set from dots. 
Sequence ~——— >})h 

Can I find m larger than Vn 

I can't see m 


not bounded. 


SET 


18; { tan x, where x =7, 27, 37, 47, 57, 67, ad 


Compute tangent 


Sin x 
tan x = 





cos x 
Put in the 77 's for the x's 
cos 77 lor-<-1; sin77— 0 
Constant 0. 


m= oO .. bounded. 


Peel, 2, 4, 8, 16, ...} 


SET 


Numbers seem to be increasing in some order. 


ea 





I think N- 


See dots s. infinity 
Can I find m? 


No. 


LA 


S12 B.N. 





Sore cr. 


eon .C. 


DEFINITION: Let E be a subset of the real numbers, R. An element, 
m, Of Ris satd to be an upper bound of E if x is equal to or less than 
m for every x in E. 


Definition is familiar. 
Wx, x€ E, xZn 
EC R 


If true, upper bound. 


PRACTICE PROBLEM: Let E be the set $ ao Ss 4} , L.e. the set consisting 


of the first four positive integers. Does E have an upper bound? 


finite set 


Bounded 


SET 2: The set of natural numbers, N (i.e. the positive integers) 


SET 


Natural numbers do not include irrational numbers 
Horizontal in space. 
Unbounded to right. 


Bounded on left. 


4: A finite set. 





i 


Biggest element (automatic) 


dpi 3 








Soros. . 


SET 5:7 1, 1/2, 1/3, =r 


1l/n first thought. 


infinite decreasing 


oes 2 | L 
<——_—_—_—_—_—_—_ 


If n very large sequence goes to Q. 


1 biggest. 


SET 6: The set of real numbers, R. 


1. line appeared — +» continuous 
2. Continuous => infinity,% no upper or lower bound. 
SET ll: The non-negative numbers. 
ine o. 1 e 2 Fd 3 & e*ee 
2. Same as above: continuous => infinity .% no upper but has lower 
bound. 
SET 13; The closed interval (3, eal 
1. interval =Jinfinite number of points 
Pant: st pape gems) 
2. biggest is 5..m25 
SET 17: {V2.V3,V4V5 VOY7, Stee f 
1. Noticed integers as written. 


114 





Si3re ac 


2. Integers unbounded. 
3. Replaced V 
4. Still increasing. 
5. Above set unbounded. 
SET 18: { tan x, where x =1, 27, 3%, 4M, 57, 67, ..p 
l. tanJ=0 
tan 27 =0 
2. Are all elements the same? 
3. Look at graph vs. computed values 
4. The elements are the same 
5. must be bounded 0 
SET 21: fu, Dp yy yy ue -} 
1. infinite set 
2. Progression 2” has no real value. 
3. unbounded. 
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S14 G.J. 


S14 G.J. 

DEFINITION: Let E be a subset of the real numbers, R. An element, 

m, Of Ris said to be an upper bound of E if x is equal to or less than 
m for every x in E. 


1. What are the relationships between (x, m, E, R)? 


2.) EC, R 
x@€ E 
me R 


x= m 


3. I had to read definition several times. 


PRACTICE PROBLEM: Let E be the set J 1, Dao a} , L.e. the set consisting 
of the first four positive integers. Does E have an upper bound? 


1. Yes, the upper bound is 4. 


This is an immediate reaction, since the set is not infinite. 
SET 2: The set of natural numbers, N (i.e. the positive integers). 


i NO. 
2. I picture mental image of above set, but continuing 
ee Soy Ay os 8 


3. Can't be upper bound&—Jinfinite. 
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S14 G.J. 
SET 4: A finite set. 


1. What is the definition of a finite set? 
2. Definition states that an upper bound and lower bound exists. 


images: 


3. Therefore has bound. 


SET 5: hv, Py 2 3 a 
step 1. I see this set as a decreasing fraction, therefore I tentatively 
conclude that the values will continue to decrease. 


step 2. Notice a pattern in the set. Numerator is constant, denominator 
is increased by l. 


1. Discovered that set =0 
2. This must be the lower bound. 
3. Examine set for greatest value. 


4. m=l. 
SET 6: The set of real numbers, R. 


1. Immediately, I remember the definition. 


2. Conclude that there is no upper bound. 
SET 11: The non-negative numbers. 
1. Define non-negative 


image; p—4-_4+-—_+-_+—4 
oa ae. 


ey 
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S14 G.J. 


2. This has same property as the set of natural numbers. 


3. State the same conclusion that there is no upper bound. 
SET 13: The closed interval [3 ' 5| 


1. Examine the interval. 
@uestate its properties: 35 x5 
3. State conclusion: m= 5 


Image: 


Ses 2 


f_—____J 
SET 17: §V2,.V3,V4V5 VE.V7, ... } 


1. Change squares into familiar digits 

V2 = 1.414 V3 = 1.732 V4 =2 ete. 
2. I notice the values increasing without bound. 
3. Out of habit, there is no upper bound. 


4. Iwas busy with the calculations so I did not see any images. 


SET 18: { Tan x, where x =, 27, 37, 47, 57, 67, ... ) 


l. Define function in more familiar terms. 
tan x = sin x/ cos x = Q 


2. Above expres-sion = 0 always, since we deal with angles where 
numerator is always Q. 


3. Conclude that the function neither increases or decreases. 


4. The upper and lower bound = QO. 
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S14 G.J. 


SET 21:4 eo eo 16. Bay 


Notice the numbers increase Imitially. 
Confirm that this will always increase without bound. 
Conclude that there is no upper bound. 


No imagery. Strictly computational. 


ES 








Soke. 


Sie R.C. 

DEFINITION: Let E be a subset of the real numbers, R. An element, 

m, Of Ris said to be an upper bound of E if x is equal to or less than 
m for every x in R. 


lL. m must be to the right of E (the elements of E). 


E 


m 
LO __ 


ilar pantomime) 


2. Note that xinm 


PRACTICE PROBLEM: Let E be the set {1, 2, 3, 44, i.e. the set consisting 
of the first four integers. Does E have an upper bound? 


1. Yes. 

2. m= 4 

3. Re-read definition to see if upper bound could be equal to an element 
of E. 


SET 2: The set of natural numbers, N (i.e. the positive integers). 


le NO. 


I realize set was infinite. 
SET 4. A finite set. 


1. Yes m= largest element. 


2. I visualize a finite set as a set of points 


e @ @ 9 
3. Took largest point as the upper bound. 
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S15 R.C. 
SET 5.) 1, oye oo 


yes, m= 1l. 
Visualization of 1 and 1/2 and 1/3. 
Realized points were going off to the left. 


—_<_—_—_—— 


tye 1/2 iL 
Since we are interested in the values on the right 


m= 1. 
SET 6: The set of real numbers, R. 


NO. 
2. Automatically realized set had no upper bound. 


No image of R. 
SET ll: The non-negative numbers. 


eNO . 


2. Image of right half of real number line came to mind to make sure 
we weren't dealing with negative numbers. 


—— AA aap A> 


0 


3. The set is infinite to the right. 
SET 13: The closed interval 3, 5| 
1. Yes m=5, no image 


2. I just saw two numbers. 


diel 





STS R.C, 


3 5 


3. Knew answer as soon as I saw it. 


SET 17: {V2V3 MiSs NEV7, ...$ 


lL. No, realized set was infinite 


2. Did not look at V 


SET 18: { tan x, where x =7, 27, 7 aT sTo7...} 


1. Oh no -- not a trignometric function. 
2. What does tangent mean? 


3. how does it relate to sin and cos? 





sin cos 
tan = - or tan = — 
cos sin 


4. I see an infinite set, but may be a trick question. 


5. I will guess unbounded. 
SET 21: a, 2,04, op 15, ae 


No, realized set was increasing. 
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S16 K.G. 


Si6°K.G. 
DEFINITION: Let E be a subset of the real numbers, R. An element, 


m, Of Ris said to be an upper bound of E if x is equal to or less than 
m for every x in E. 


1. Had to read the definition a number of times before I even had the 
vagous idea of what an upper bound was. 
2. I could not determine relationship between x and m. 
3. Remembered to use Venn diagrams. 
U 


& 


4. I now saw that m had to be in R and had to be greater or equal to x. 


PRACTICE PROBLEM: Let E be the set te ZG 3% at, i.e. the set consisting 
of the first four positive integers. Does E have an upper bound? 


1. This is not type of problems I expected, although I should have 
from definition. 

2. It seems obvious. 

3. I know 5 or greater is an upper bound, but is 4? 


4. Yes. 


SET 2: The set of natural numbers, N (i.e. the positive integers) 





1. Read problem 


2. I believe that no upper bound exists because the numbers go to infinity. 
— 


3. I remember from a previous course the conflict about 0 being in the 
natural numbers. 
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S16 K.G. 


SET 4: A finite set. 


I cannot see an m in the set. 
I re-read the definition. 


Since no specific values are given -- how can I determine an 
upper bound. 


I am confused. I need numbers. 


I guess no upper bound. 


eee. { 1, 1/2, 1/3, a5 


ee 


Ze 


First noticed an infinite set and concluded no upper bound. 


but, wait, it is decreasing -» the largest number must be the 
upper bound = l. 


SET 6: The set of real numbers, R. 


ive 


Zi 


3. 


I imagine all types of numbers 
(rational, irrational, natural, etc.) 
I see it going on forever. 


No upper bound. 


SET ll: The non-negative numbers. 


a 


7 age 


Had trouble figuring out non-negative. First thought numbers to 
the left of zero. 


e—— 
Then thought of positive numbers, so no upper bound. 
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S16 K.G. 


13: The closed interval [3 5]. 


sel 


l. 


Immediately thought upper bound. 


SET 17: IV2V3 Mav's Vev7, <a } 


SET 


First noticed square root sign. 
Then noticed progression with no upper bound. 
Then asked if square root would increase to SO 


Decided it would, so no upper bound. 


18: } Tan x, where x =7, 27, 34, 47, 57, 67, ve} 


Originally thought no upper bound because of infinite progression. 
But then realized that the value for tan = 0 at the given x values. 


There is a upper bound. 


21: f1, 2, 4, 8, 16, Lead 


SET 


Rapidly increasing numbers, so probably no upper bound. 


No mental imagery. 
Saw a geometric series, but of no importance. The set is infinite. 


12) 





Sg eh it: 


ol 7 en. T. 
DEFINITION: Let E be a subset of the real numbers, R. An element, 


m, Of Ris said to be an upper bound of E if x is equal to or less than 
m for every x in E. 


1. Discovered what the definition talks about, and what elements were 
in it. Then tried to relate the elements to each other. 


2. m must be in R and greater or equal to x, which is in E. 
E<— x=m wR 


PRACTICE PROBLEM: Let E be the set fl, 2 je jae 3, i.e. the set consisting 
of the first four positive integers. Does E have an upper bound? 
1. From the definition 

E<— 1, 2, 3, 4 m —>R 


2. Therefore any number = 4 is the upper bound. 
SET 2: The set of natural numbers, N (i.e. the positive integers). 


1. First drew mental image of the set 


d ie 2 eo, 4: ay and saw a number line going into infinity. 


6_a__a._ 2 __ 2 


2. I decided that there is no m which is greater or equal to the 
largest x,e. nO upper bound. 


SET 4: A finite set. 





1. What is meant by a finite set? 
A specific number of points ——= countable. 


For example: 4 ee at, 


126 





Sivn. . « 


2. Since the set is countable it has an upper limit. 


3. Remembered the term countable from discreet math or something. 


SET 5:f1, 1/2, 1/3, sed 


1. I first tried to find a pattern, but could not find one. 
2. Recognized "...". 


3. Infinite set and therefore not bounded. 


SET 6: The set of real numbers, R. 





1. I ama little confused -- we are working with the real numbers 
(from definition). 


2. If a set is a subset of itself then there is no upper bound. 


image: 


SET ll: The non-negative numbers. 


1. First read "negative numbers". A line which moves to the left 
instead of right. 
2. Read again and saw line moving to the left 
No 
Sth —— 


Se No. 


SET 13: The closed interval 3, s}. 


1. Immediately related this to problem before. 


Ze 





Sey £1’. 


Closed interval - finite - upper bound. 


2. Used no imagery except noticed 8 ]- 


SET 17: $2.3 iS MEV7, ae 3 


i. No. 


2. Again "..." was seen. Infinite set — no upper bound, 


SET 18: f Tan x, where x =, 27, 37, 47, 57, 67, +} 


1. I always had problems with tangent function. 
2. The only images I see is the words, 
SIN Cos 


but don't know how they relate to the 7 values. 
SET 21: {1, 2, 4, 8, 16,-.-} 


1. tLook at set. 
2. Notice doubling —ssome type of geometric progression. 


3% No upper bound/ no imagery. 
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SL Giis i. 


S18 Ne Lae 
DEFINITION: Let E be a subset of the real numbers, R. An element, 
m, Of Ris said to be an upper bound of E if x is equal to or less than 
m for every x in E. 
1. I remember a definition similar to this for the upper bound. 
a. X—pE —>R and m—>R 
must hold that 


m=x 


PRACTICE PROBLEM: Let E be the set {1, oe a}, i.e. the set consisting 
of the first four positive integers. Does E have an upper bound? 
1. I visualize the definition 
1, 2, 3, 4—p> E —DR (realsh 
m=1, 2, 3, 4 
2. Reals are easy to define. 
3. E was also easy to define. 
4. Because I'd made an equation for m, I could plug the elements of 
E into x to convince myself that 4 was the upper bound. 


SET 2: The set of natural numbers, N (i.e. the positive integers). 


1. The numbers are positive and move to right. 





L223 4 => 


2. Nothing can be to right of all of them, therefore upper bound exists. 


SET 4: A finite set. 


l. Same as practice problem 
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Slew. lL. 


2. Yes. 


3. Thought about answer immediate. 


SET 5; aT 1/2 8073 = 


1. Another sequence. 

ae 1, 1/2, 1/3, ... —s» E—aA R 
nee ly 1/2987 3 pees 

3. Descending, therefore m exists. 


4. Upper bound is anything equal to or greater than l. 
SET 6: The set of real numbers, R. 


1. Clearly No!!! 
2. It is just like N, moves to the right. 
no image 


no upper bound 
SET 11: The non-negative numbers. 


1. The word non-negative confusing. 
2. Must be positive numbers. 

As before move off to the right. 
3. No image. 


No upper bound. 


SET 13: The closed interval ie 5 | 
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1. Knew the answer before w-iting. 
2. Thought that whether the interval is open or closed is irrelevant. 
SET 17: {23M M5 87, ee } 
1. Another sequence. 
2. Square root is unbounded —-3Wn = © 
3. Therefore no upper bound exist. 
4. No image. 
SET 1s: { Tan x, where x =77, 27, 37, 477, 57, 67, vee fe 
1. This made me think about high school math. 
2. Sequence. 
tan = y/x = 0 
3. Upper bound exists. 
set 21: {1, 2, 4, 8, 16, ...}. 
1. Some type of geometric sequence. 
2. Obviously fast disappearing to right. 
3. No upper bound. 


gis J.L. 


No image. 


lesa: 





S19 L.N. 
S19 L.N. 
DEFINITION: Let E be a subset of the real numbers, R. An element, 


m, O£f Ris said to be an upper bound of E if x is equal to or less than 
m for every x in E. 


ne 


Ze 


je¥>y fs} 8 


PRACTICE PROBLEM: Let E be the set}{1, 2, 3, 4}, i.e. the set consisting 
of the first four positive integers. Does E have an upper bound? 


T. ee nn Gin 4 
12 3 4 


Look to right for highest order number. 


SET 2: The set of natural numbers, N (i.e. the positive integers). 


od [-—_a—_-2-—_+—f § etc. 


i Z 3 


2. Asked myself if natural was correct tern. 
Drew line, checked to make sure I had not included zero. 


3. Where is set l? 


SET 4: A finite set. 


le ee Roane 
“ats ; pao > eecae 





132 





Sian .N. 


2. What does he want? 
I think of points in line or an array. 


3. Since only specifted number of points, upper bound exists and 
must be n. 


4. Where is set 3? 


SET 5: fi. Peis sd 


l. Ratternat-numbers- -- not all there. 


infinite set 


{in n=1 to & 


i, o] =o 
2. Decreasing, largest value is upper bound. 


3. Are these questions or stimuli? 


Are they random? Is that why the set numbers vary? 


SET 6: The set of real numbers, R. 


a, é—_|_—> Re 


soltdeline 





usually think of Re at end of line. 


2. Infinite, thus unbounded. 


SET ll: The non-negative numbers. 


, p20 


Ope ee, ae eG. 


2. Include zero this time. 


coo 





Sif LN. 


SET 13: The closed interval [3, 5 |. 
ib, — 
3 5 
2. Open interval uses ( l1- 
Closed interval uses C | 


oy 95 


SET 17: f V2,V3, VEVE VE N7, ee $ 


1. Integers @ 1 


{vm n=2 —»? & 


2. Why so cryptic? 
I am looking for small details. 


3. No bound —— infinity. 
SET 18: as Tan x, where x= 7, 27, BF, 4d, 57, 6%, .«.- 5 


1. What goes on here? 

2. What is he trying to do with this set? 

3. What can he hope to gain from this experiment? 
Looks like a psych. experiment. 


4. Why so much blank paper? 


SET 21: fu, oe aes sich 4 


{23 NeOrl 2, «es 


2. Infinite set. 
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S20, 0. 


S20 T.M. 


DEFINITION: Let E be a subset of the real numbers, R. An element, m, 
of Ris said to be an upper bound of E if x is equal to or less than 
m for every x In E. 


1. Very confusing. I start to panic. 


2. Relate subset E to point m. 
Te 
3. Look at “less than" or "equal to" part of definition. 


PRACTICE PROBLEM: Let E be the set $1, 2, 3; at, t.e. the set consisting 
of the first four integers. Does E Have an upper bound? 


1. Yes. 

SET 2: The set of natural numbers, N (i.e. the positive integers). 
1. Saw this: 

2. Not paying attention to the left, but infinite in both direction 


3. Concluded no. 


3. "Is N bounded?" No. 
SET 4: A finite set 


1. I don't see any numbers. 


2. What happens if the finite set has infinite points 


[(—9, + 90] 
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Bi 


$20.7 .M. 


How can I assign an m to this. 


SET 5: J ee 25 aly 3. veel 


1. I see a sequence going to infinity 

a______-_--_> 

i 
2. Therefore must be bounded. 
SET 6: The set of real numbers, R. 
1. Automatic reaction to infinite set, thus could not define m. 
2. No mental image. 
3. Definition mentions R how does it relate to this set? 
SET 11: The non-negative numbers. 
1. Half-line in mind —— y positive. 
2. Here, I realized no upper bound. 
SET 13: The closed interval Be, 5 | 
1. First see the word interval, indicates no upper bound. 
2. Infinite number of points. 

Qe ) 

3 = No oe 
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SZenr.M. 


SET 17 :$V2,V3,V4.V5 V6 V7, oy -} 


1. No upper bound. 
2. I changed to more conventional numbers 
$224, 12492, 2, etc. 

3. Then I was convinced the serftes ascended and was infinite. 
SET 18:f Tan x, where x =7, oF. 377, 47, 57, 67, vo} 
lL. No upper bound. 
2. Like the previous problem -— I am convinced the series is ascending 

and is infinite. 
3. No mental image. 
SET 21: Jan yaa) icy lee — 
1. The progression 1S ascending very rapidly to the right 

Fast 
eo 

Zoe Leis infinite Ps no upper bound. 


137 


1 





SZ. C. 


Sei A.C. 
DEFINITION: Let E be the subset of the real numbers, R. An element, 


m, Of Ris said to be an upper bound of E if x is equal to or less than 
m for every x in E. 


1. Image: 


a 


2. I try to relate the elements on the line. 


Draw: 
upper bound 
> 
——-* 
m 
E ! 


PRACTICE PROBLEM: Let E be the set oa, io a} , 1.e. the set consisting 
of the first four positive integers. Does E have an upper bound? 

l. I see 4 points or elements. 

2. Hazy, dark with white circles. 

B. Finite. 


ae Les. 
SET 2: The set of natural numbers, N (i.e. the positive integers). 


1. Got side-tracked on definition. 
2. Remember the term "countably infinite". 
The set numbers does approach infinity 


3. I visualize myself counting the positive integers and saw there 
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SZi Av. 


would be no end to the counting. 


4. 


No upper bound. 


SET 4: A finite set. 


Hiss 


2. 


S. 


ee 


2. 


This does not seem to fulfill all the requirements of definition. 
But finite means an ending. 
If there is an end then there must be an m greater than the end. 


Its finite se therefore upper bound. 


SET 5: 2, 23), -oeh 


The infinite series converges to 0. 
The answer is the largest element of the set. 


m= l or greater. 


SET 6: The set of real numbers, R. 


i, 


are 


I visualize a single straight line with no end in either direction. 





I conclude there is no bound in either direction. 


SET ll: The non-negative numbers. 


What are non-negative numbers? 
Answer: positive numbers. 
Positive numbers go to infinity. 
Therefore no bound (upper). 


No image. 
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moleA.C. 


SET 


SET 


13: The closed interval 3, 5 


Closed interval is finite. 
Is it? 


Draw: 


3 Ss 
japaneses] 
No, interval is not finite. 


But closed means endpoint. 


Yes, m=5 


17: $ V2,V3,V4,V5 V6,V7, 8 


SET 


Tried to determine if V of numbers was increasing or decreasing. 
Saw the dots. 

I better determine if increasing or decreasing. 

Increasing. 

The set is infinitely large. 


No upper bound. 


18: 4 Tan x where x =, 27, 37, 47, SI, GF, vee} 


Tried to determine if tan x is increaSing or decreasing. 
Sin x 

I remember: tan x = 
cos x 





But can't relate the J values to them. 


Saw the-dots. 
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The set is increasing. 


The set is infinitely large. 


6. No upper bound. 

SET 21: h1, ayy ee Ne ee le. 

1. As before, the set increases --- 
2. Saw the dots. 

3. No upper bound. 

4. No image, because obvious. 
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22 edie Leg 


mae J. le 
DEFINITION: Let E be a subset of the real numbers, R. An element, 


m, 1S Said to be an upper bound of E if x is equal to or less than 
m for every x in E. 


1. E}R} 

a OS ={x is an element } 

3. ma x —SR 

4. The meaning of upper bound is ambiguous. I hope to figure it out 


after doing some problems. 


PRACTICE PROBLEMS: Let E be the setq7l, 2, 3, a}, It.e. the set consisting 
of the first four positive integers. “ Does E have an upper bound? 

ier = 1, 2, 3, 4 

2. 1, 2, 3, 4 are positive. 

3. Read definition. 

4. where does m fit? 

5. mis a real, so is x. 

6. Aha, m=x, therefore 4 or greater. 


7. Let's try next problem. 
SET 2: The set of natural numbers, N (i.e. the positive integers). 


1. ERS i295) cone 
image: Think of positive integers and natural numbers. 
2. Read definition. 


3. The set goes to infinity. Does that mean if m = ®it is the upper 
bound? 
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S2cdei. 


4, Infinity is not a real number ,°, no upper bound. 


5. I am still confused with definition. 
SET 4: A finite set. 


1. This is the same as the practice problem. 
2. Yes, upper bound. 


3. I didn't use definition or pictures for this set. 


ea) 1, 1/2, 1/3, ...}. 


1. This set is also like the preaious problem. 
2. Turn to definition. 
3. Confused. 


4. Draw. 


o—_—__—__—__Y 
F ‘hh = 


-” — 


5. It is infinite and therefore there must be no m, no upper bound. 
SET 6: The set of real numbers, R. 


1. Infinite series in both direction, not just to the right. Zero 
as. center. 


2. image: 


a 
3. Getting the hang of definition. 


If no m or infinite no upper bound. 


SET ll; The non-negative numbers. 
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S22eu. 1. 


1. Infinite series to the left (mo image). 
2. Same as previous problem. 
No m and infinite means no upper bound. 


3. The problems are going quicker now. 
SET 13: The closed interval Bi 5 | 


l. This is finite. 


2. Upper bound. 
SET 17:3V2,V3,V4,V5 ,V6,V7, aah 
l. Infinite series. 


images: / 


2. numbers seem to be increasing. 


3. As before, no m, no upper bound. 


SET 18: { Tan x, where x =7, 27, 37, 47, 57, 67, } 


1. Trig function are cyclic 


Image: 


3. Since infinite sequence, graph of tan function goes to infinity 
somewhere. 
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eco... 


4. No upper bound. 


SET 21: 1, DearS 516 - — 


1. Saw familiar pattern, oy 


2. Saw dots. 
3. Quickly decided it was Infinite. 
4. No upper bound. 


5. No imagery. 
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oe2o) 0.5. 


23 J.B. 


DEFINITION: Let E be a subset of the real numbers, R. An element, 
m, Of Ris said to be an upper bound of E if x is equal to or less than 
m for every x in E. 


ls 


First I have to think what the term subset means. 
I think about a set, which is a collection of things: 
(i.e. red, blue, green) 
A Venn diagram comes to mind, using real numbers as the universe. 
R 


A subset, must therefore be a small set, 
within R. 





R 


©) 


Looking at the Venn diagram, I think about the upper bound as 
being everything in the universe R with the exception of the 
area within the small circle. 


Y 


LLL 


PRACTICE PROBLEM: Let E be the set fi, pid; 4} , 1.e. the set consisting 
of the first four positive integers. Does E have an upper bound? 


bs 


Ze 


This subset contains a finite set of numbers. 


They are increasing in size. 


L233. 4 
—§—_t—_o—_-¢ 
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525-0) .B. 


3. Since it is finite the largest number is 4. The upper bound = 4. 


SET 2: The set of natural numbers, N (i.e. the positive integers). 





1. First thing that comes to mind is -=- what is a natural number. 
2. The series is increasing 


Onl si Zase. 3 
-+—_+—_e— > > 


3. Its an infinite series going to infinity (© ) 


4. No upper bound. 


SET 4; A £inite set 





1. what is a finite set? 


2. Its a specified number of things. 


a 9 ao £ @ 2. 24 


e Ae 


3. Therefore it has an upper bound. 


SET 5; a 1/2, 1/3, aa 


1. Look at the dots. indicating a infinite set. 
2. Infinite sets do not have upper bounds. 
3. Looking at the numbers they are decreasing going towards Q. 


173 1/2 


4. Upper bound of l. 
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S25 0.5. 


SET 6: The set of real numbers, R. 


SET : 


What is the set of real numbers. 


Remember the Venn diagram TIT used in the definition. 


No upper bound, since it goes to infinity. 


ee The non-negative numbers. 


1. This brings back set 2 (natural # - positive integers) 
0 => co 
2. It goes to infinity, therefore no upper bound. 
SET 13: The closed interval Ee 5] 
1. Immediate reaction of upper bound of 5, since it is a finite 
set. 
5 5 
2 w 
SET 17: {V2,¥3,V4,V5,V6 7, roof 
1. First looked at dots, indicating an infinite set, therefore 
no upper bound. 
ye. V4 = 2 
Vo = 3 
3. Indicates the larger the number gets -- the larger the 


square of it. 
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S2o00 -=. 


No upper bound goes to © 


18: lane <nwaere. «= 2, 3, 4,5 , © », ««-s 


SET 


Had to remember what tangent ts. 


Drew picture to figure it out. 


sine sin 
an 


cos 
Cos 


What where the values of sin & cos at 7/ ae” 7, etc. 


W D7) 
74 aw 
SIN COS 
cos -l 
Sin 2 0 
ee ees 2 ete TS 


Therefore the set is Oras, (O50 fescs 


Eventhough infinite set the numbers are constant Q. Therefore 
upper bound of 0Q. 


SET 


Bi: § lie 2ucimayers, ed} 


Its an infinite series -- does it have an upper bound. 
Equation is 2 n-1) 


When n becomes so large the -1 will have no effect. 


Increasing set and since n can go to infinity -- no upper bound. 
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ie 


S2a0oGM. 


This is true for all finite sets. 


SET 5: $1, 23. ved 


Ee 


Ze 


oe 


Saw the form l/n. 
Looked at consecutive pairs; saw that denominator was increasing. 
Infinite set and therefore upper bound. 


(no image) 


SET 6: The set of real numbers, R. 


i 


2. 


From memory R has no upper bound. 
I don't think I used the image. 


I hesitate to use too many images because they are unreliable. 


SET ll: The non-negative numbers. 


lle 


Ze 


I think what it means to be non-negative, getting an image 
of = 0. Again I see an infinite set, and know that it is 


unbounded. 


I note that it is bounded below. 


SET 13: The closed interval 3 ; 5 


i 


one 


An interval means it is infinite, but also it is bracketed on 


both sides 
E oO | 


In this problem the infinite set is bounded by 3 and 5, therefore 
it has an upper bound. 


150 





S24 S.M. 


S24 S.M. 
DEFINITION: Let E be a subset of the real numbers, R. An element, 


m, Of Ris said to be an upper bound of E if x is equal to or less than 
m for every x in E. 


Ls mé.R 
Ze 6X SS OM 
3. The above equation is what I thought about when reading the 


definition. 


PRACTICE PROBLEM: Let E be the setyl, 2, 3, 4g, i.e. the set consisting 
of the first four positive integers. Does E have an upper bound? 
1. 4 elements in the set (finite set). 


2. Therefore there is an upper bound. 


SET 2: The set of natural numbers, N (i.e. the positive integers). 


1. I focus on a particular set of numbers. 

ions 4364 etc 
2. I can't tell the last element of E, since the number go to infinity. 
3. I wonder if these do have upper bound? 


4. I can't see an end to this set - so no upper bound. 
SET 4; A finite set. 


1. Concrete at end and beginning. 


Draw: 
$a, bo G, oO, 6, 2, gt 


If I use above set where a b c d é £ g, then 


g is the upper bound. 


St 





3. 


4. 


S24 S.M. 


Confused, what would be the answer if the question was: 
The closed interval [- 2° ,t on | 


My answer for this problem is upper bound. 


SET 17:$ V2,V3,V4,V5,V6V7, " -¢ 


SET 


No, realized set was infinite. 

1. Ignored Vv 

2. Saw dots. 

SET 1s: § Tan x, where x =7/, 2%, aA 47, 57, 67, Bee 


I tried to visualize graph of tan x: 





Disregarded graph, because it didn't tell me what the values 
were at 7 , 27, etc. 


Went to mathematical form: 
sin 


tan = ==- 
cos 


Not sure what the values are for and 277 
See dots. 
The set is infinite,% no upper bound. 


I did not really solve the tan problem. 


Dili £4 Dera Vala . --. 


No, realized an infinite geometric increasing set. 
no image (automatic) 


Way 
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